Checkpoints

[eduardocarvp]

Feature request

Save/load/average checkpoints.

What is the expected behavior?

What is motivation or use case for adding/changing the behavior?
Smarter early stopping and possibly better generalization on predictions.

How should this be implemented in your opinion?
Good source of inspiration here: https://github.com/Qwicen/node/blob/master/lib/trainer.py

Are you willing to work on this yourself?
yes

[Optimox]

@eduardocarvp I thinkg with Stochastic Weight Averaging natively available in PyTorch 1.6 we could do this in a very simple way and add a parameter for when to start SWA: https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/