Plans for pytorch_optimizer v3

[kozistr]

In pytorch-optimizer v3, loss function will be added. So, finally, the optimizer & lr scheduler & loss function are all in one package.

Feature

• support at least 60 optimizers
• support at least 10 objectives
• support bitsandbytes (& 4-bit optimizers)

Refactor

• Organize utils

Docs

• Organize documentation
• Support contribution guide (implementation, test, etc...)
• Add issue templates
• Migrate to mkdocs
• Create Q&A page
• Benchmark on ImageNet

Test

• Organize test cases

[redknightlois]

IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
https://arxiv.org/abs/1903.12141