Momentum optimizer
OptimizerMomentum.RdThis class defines momentum.
Super class
vistool::Optimizer -> OptimizerMomentum
Methods
Method new()
Creates a new instance of this [R6][R6::R6Class] class.
Usage
OptimizerMomentum$new(
objective,
x_start,
lr = 0.01,
momentum = 0.9,
id = "Momentum",
print_trace = TRUE
)Arguments
objective(`Objective`) The objective to optimize.
x_start(`numeric()`) Start value of the optimization. Note, after the first call of `$optimize()` the last value is used to continue optimization. Get this value with `$x`.
lr(`numeric(1)`) Step size with which the update is multiplied.
momentum(`numeric(1)`) Momentum value.
id(`character(1)`) Id of the object.
print_trace(`logical(1)`) Indicator whether to print the status of `$optimize()`.
Method optimize()
Optimize `steps` iteration.
Usage
OptimizerMomentum$optimize(
steps = 1L,
stepSizeControl = function(x, u, obj, opt) return(1),
minimize = NULL
)Arguments
steps(`integer(1)`) Number of steps/iterations.
stepSizeControl(`function()`) A function with arguments `x` (the old input value), `u` (the upate generated by `$update()`), `obj` (the objective object), and `opt` the optimizer as `self`. Default simply returns 1 to include the update as is. Note that the update `u` is calculated based on the formula with learning rate, e.g., for GD x_new = x_old + lr * g, with update u = lr * g. Step size control takes x_old and u (with included learning rate) and calculates a value that is used to extend or compress u for the final update.
minimize(`logical(1)`) Indicator to whether minimize or optimize the objective. The default (`NULL`) uses the option defined in `objective$minimize`.
Method update()
Calculate the update for `x`