pypose.optim.scheduler.StopOnPlateau

class pypose.optim.scheduler.StopOnPlateau(optimizer, steps, patience=5, decreasing=0.001, verbose=False)

A scheduler to stop an optimizer when no relative loss ‘decreasing’ is seen for a ‘patience’ number of steps.

Parameters

• optimizer (Optimizer) – Wrapped optimizer.

• patience (int) – Number of steps with no loss ‘decreasing’ is seen. For example, if patience = 2, then it ignores the first 2 steps with no improvement, and stop the optimizer after the 3rd step if the loss has no decerasing. Default: 5.

• decreasing (float) – relative loss decreasing used to count the number of patience steps. Default: 1e-3.

• verbose (bool) – if True, prints a message to stdout for each step. Default: False.

Note

The users have two options to use a scheduler. The first one is to call the step() method for multiple times, which is easy to be extended for customized operation in each iteration. The second one is to call the optimize() method, which interally calls step() multiple times to perform full optimization. See examples below.

optimize(input, target=None, weight=None)

Perform full optimization steps.

Parameters

• input (Tensor/LieTensor or tuple of Tensors/LieTensors) – the input to the model.

• target (Tensor/LieTensor) – the model target to optimize. If not given, the squared model output is minimized. Defaults: None.

• weight (Tensor, optional) – a square positive definite matrix defining the weight of model residual. Default: None.

The above arguments are sent to optimizers. More details go to pypose.optim.LevenbergMarquardt or pypose.optim.GaussNewton.

Example

>>> class PoseInv(nn.Module):
...
...     def __init__(self, *dim):
...         super().__init__()
...         self.pose = pp.Parameter(pp.randn_SE3(*dim))
...
...     def forward(self, input):
...         return (self.pose @ input).Log().tensor()
...
>>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
>>> input = pp.randn_SE3(2, 2).to(device)
>>> invnet = PoseInv(2, 2).to(device)
>>> strategy = pp.optim.strategy.Constant(damping=1e-4)
>>> optimizer = pp.optim.LM(invnet, strategy=strategy)
>>> scheduler = pp.optim.scheduler.StopOnPlateau(optimizer, steps=10, \
>>>                     patience=3, decreasing=1e-3, verbose=True)
...
>>> scheduler.optimize(input=input)
StopOnPlateau on step 0 Loss 5.199298e+01 --> Loss 8.425808e-06 (reduction/loss: 1.0000e+00).
StopOnPlateau on step 1 Loss 8.425808e-06 --> Loss 3.456247e-13 (reduction/loss: 1.0000e+00).
StopOnPlateau on step 2 Loss 3.456247e-13 --> Loss 1.525355e-13 (reduction/loss: 5.5867e-01).
StopOnPlateau on step 3 Loss 1.525355e-13 --> Loss 6.769275e-14 (reduction/loss: 5.5622e-01).
StopOnPlateau: Maximum patience steps reached, Quiting..

step(loss)

Performs a scheduler step.

Parameters

loss (float) – the model loss after one optimizer step.

Example

>>> class PoseInv(nn.Module):
...
...     def __init__(self, *dim):
...         super().__init__()
...         self.pose = pp.Parameter(pp.randn_SE3(*dim))
...
...     def forward(self, input):
...         return (self.pose @ input).Log().tensor()
...
>>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
>>> input = pp.randn_SE3(2, 2).to(device)
>>> invnet = PoseInv(2, 2).to(device)
>>> strategy = pp.optim.strategy.Constant(damping=1e-4)
>>> optimizer = pp.optim.LM(invnet, strategy=strategy)
>>> scheduler = pp.optim.scheduler.StopOnPlateau(optimizer, steps=10, \
>>>                     patience=3, decreasing=1e-3, verbose=True)
...
>>> while scheduler.continual:
...     loss = optimizer.step(input)
...     scheduler.step(loss)
StopOnPlateau on step 0 Loss 9.337769e+01 --> Loss 3.502787e-05 (reduction/loss: 1.0000e+00).
StopOnPlateau on step 1 Loss 3.502787e-05 --> Loss 4.527339e-13 (reduction/loss: 1.0000e+00).
StopOnPlateau on step 2 Loss 4.527339e-13 --> Loss 7.112640e-14 (reduction/loss: 8.4290e-01).
StopOnPlateau on step 3 Loss 7.112640e-14 --> Loss 3.693307e-14 (reduction/loss: 4.8074e-01).
StopOnPlateau: Maximum patience steps reached, Quiting..