Exploring the new optimiser: Adan

This repo trains an image classifier with ADAN (Adaptive Nesterov momentum algorithm) and compares the results with standard Adam.

• We use @lucidrains PyTorch implementation inside a fastai training loop.
• You can also find a fastai's style Adan implementation by Benjamin Warner in fastai_adan.py

Results

Go to the W&B report and check the training results