Skip to contents

Momentum optimizer

OptimizerMomentum.Rd

This class defines momentum.

Super class

vistool::Optimizer -> OptimizerMomentum

Active bindings

momentum

(numeric(1)) Momentum of the algorithm.

Methods

Public methods

Inherited methods

Method new()

Creates a new instance of this R6 class.

Usage

OptimizerMomentum$new(
  objective,
  x_start,
  lr = 0.01,
  momentum = 0.9,
  id = "Momentum",
  print_trace = TRUE
)

Arguments

objective

(Objective)
The objective to optimize.

x_start

(numeric())
Start value of the optimization. Note, after the first call of $optimize() the last value is used to continue optimization. Get this value with $x.

lr

(numeric(1))
Step size with which the update is multiplied.

momentum

(numeric(1))
Momentum value.

id

(character(1))
Id of the object.

print_trace

(logical(1))
Indicator whether to print the status of $optimize().

Method optimize()

Optimize steps iteration.

Usage

OptimizerMomentum$optimize(
  steps = 1L,
  step_size_control = function(x, u, obj, opt) {
     return(1)
 },
  minimize = NULL
)

Arguments

steps

(integer(1))
Number of steps/iterations.

step_size_control

(function()) A function with arguments x (the old input value), u (the update generated by $update()), obj (the objective object), and opt the optimizer as self. Default simply returns 1 to include the update as is. Note that the update u is calculated based on the formula with learning rate, e.g., for GD x_new = x_old + lr * g, with update u = lr * g. Step size control takes x_old and u (with included learning rate) and calculates a value that is used to extend or compress u for the final update.

minimize

(logical(1)) Indicator to whether minimize or optimize the objective. The default (NULL) uses the option defined in objective$minimize.

Method update()

Calculate the update for x

Usage

OptimizerMomentum$update(lr, mom)

Arguments

lr

(numeric(1)) The learning rate.

mom

(numeric(1)) The momentum.

Method clone()

The objects of this class are cloneable with this method.

Usage

OptimizerMomentum$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.