pymc.apply_momentum

pymc.apply_momentum(updates, params=None, momentum=0.9)

Return a modified update dictionary including momentum.

Generate update expressions of the form:

• velocity := momentum * velocity + updates[param] - param

• param := param + velocity

Parameters:

updates: OrderedDict

A dictionary mapping parameters to update expressions

params: iterable of shared variables, optional

The variables to apply momentum to. If omitted, will apply momentum to all updates.keys().

momentum: float or symbolic scalar, optional

The amount of momentum to apply. Higher momentum results in smoothing over more update steps. Defaults to 0.9.

Returns:

OrderedDict

A copy of updates with momentum updates for all params.

See also

momentum

Shortcut applying momentum to SGD updates

Notes

Higher momentum also results in larger update steps. To counter that, you can optionally scale your learning rate by 1 - momentum.