Momentum optimizer
OptimizerMomentum.RdThis class defines momentum.
Super class
vistool::Optimizer -> OptimizerMomentum
Methods
Method new()
Creates a new instance of this [R6][R6::R6Class] class.
Usage
OptimizerMomentum$new(
objective,
x_start,
lr = 0.01,
momentum = 0.9,
id = "Momentum",
print_trace = TRUE
)Arguments
objective(`Objective`)
The objective to optimize.x_start(`numeric()`)
Start value of the optimization. Note, after the first call of `$optimize()` the last value is used to continue optimization. Get this value with `$x`.lr(`numeric(1)`)
Step size with which the update is multiplied.momentum(`numeric(1)`)
Momentum value.id(`character(1)`)
Id of the object.print_trace(`logical(1)`)
Indicator whether to print the status of `$optimize()`.
Method optimize()
Optimize `steps` iteration.
Usage
OptimizerMomentum$optimize(
steps = 1L,
stepSizeControl = function(x, u, obj, opt) {
return(1)
},
minimize = NULL
)Arguments
steps(`integer(1)`)
Number of steps/iterations.stepSizeControl(`function()`) A function with arguments `x` (the old input value), `u` (the update generated by `$update()`), `obj` (the objective object), and `opt` the optimizer as `self`. Default simply returns 1 to include the update as is. Note that the update `u` is calculated based on the formula with learning rate, e.g., for GD x_new = x_old + lr * g, with update u = lr * g. Step size control takes x_old and u (with included learning rate) and calculates a value that is used to extend or compress u for the final update.
minimize(`logical(1)`) Indicator to whether minimize or optimize the objective. The default (`NULL`) uses the option defined in `objective$minimize`.
Method update()
Calculate the update for `x`