Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exponential Moving Average of Weights (EMA) #138

Closed
wants to merge 1 commit into from
Closed

Exponential Moving Average of Weights (EMA) #138

wants to merge 1 commit into from

Commits on May 5, 2021

  1. Exponential Moving Average of Weights (EMA) 

    EMA as used in "Fast and Accurate Model Scaling" to improve accuracy.
Note that EMA of model weights is nearly free computationally (if not
computed every iter), hence EMA weights area always computed/stored.
Saving/loading checkpoints has been updated, but the code is backward 
compatible with checkpoints that do not store the ema weights.

Details:
-config.py: added EMA options
-meters.py: generalized to allow for ema meter
-net.py: added update_model_ema() to compute model ema
-trainer.py: added updating/testing/logging of ema model
-checkpoint.py: save/load_checkpoint() also save/load ema weights
    @pdollar
    pdollar committed May 5, 2021
    Configuration menu
    Copy the full SHA
    ee770af View commit details
    Browse the repository at this point in the history