Demon
Demon: Improved Neural Network Training with Momentum Decay
A schedule for momentum decay which greatly improves convergence rate on various network architectures, for SGD-M and Adam optimizers.
James Leopore
from a laptop in Sunnyvale
from a laptop in Sunnyvale
Demon: Improved Neural Network Training with Momentum Decay
A schedule for momentum decay which greatly improves convergence rate on various network architectures, for SGD-M and Adam optimizers.