V0.2.6-MogaNet-ImageNet-Weights
·
70 commits
to main
since this release
A collection of weights and logs for image classification experiments of MogaNet on ImageNet-1K (download). You can download all files from Baidu Cloud (z8mf) at MogaNet/Classification_OpenMixup.
- We train MogaNet for 100 and 300 epochs according to the RSB A3 and DeiT settings on ImageNet-1K. Note that * denotes the refined training setting of lightweight models with 3-Augment. Refer to the Appendix of MogaNet for more training details.
- The best top-1 accuracy of image classification in the last 10 training epochs is reported for all experiments. Note that we report the classification accuracy of EMA weights for MogaNet-S, MogaNet-B, and MogaNet-L.
- As for evaluation experiments of the pre-trained weights, you can test them with
tools/dist_test.shfor the classification performance or fine-tune them on downstream tasks by only loading the encoder weights, e.g., COCO detection and ADE20K segmentation. - Warning of
attn_force_fp32: During fp16 training, we force to run the gating functions with fp32 to avoid inf or nan. We found that if we useattn_force_fp32=Trueduring training, it should also keepattn_force_fp32=Trueduring evaluation. This might be caused by the difference between the output results of usingattn_force_fp32or not. It will not affect performances of fully fine-tuning but the results of transfer learning (e.g., COCO Mask-RCNN freezes the parameters of the first stage). We set it to true by default in OpenMixup while removing it in MogaNet implementation. For example, you can use moga_small_ema_sz224_8xb128_ep300.pth withattn_force_fp32=Truewhile using moga_small_ema_sz224_8xb128_no_forcefp32_ep300.pth withattn_force_fp32=False.
Image Classification on ImageNet-1K
| Model | Pretrain | Setting | resolution | Params(M) | Flops(G) | Top-1 (%) | Config | Download |
|---|---|---|---|---|---|---|---|---|
| MogaNet-XT | From scratch | DeiT | 224x224 | 2.97 | 0.80 | 76.5 | config | model | log |
| MogaNet-XT | From scratch | DeiT | 256x256 | 2.97 | 1.04 | 77.2 | config | model | log |
| MogaNet-XT* | From scratch | DeiT-3 | 256x256 | 2.97 | 1.04 | 77.6 | config | model | log |
| MogaNet-T | From scratch | DeiT | 224x224 | 5.20 | 1.10 | 79.0 | config | model | log |
| MogaNet-T | From scratch | DeiT | 256x256 | 5.20 | 1.44 | 79.6 | config | model | log |
| MogaNet-T* | From scratch | DeiT-3 | 256x256 | 5.20 | 1.44 | 80.0 | config | model | log |
| MogaNet-S | From scratch | DeiT | 224x224 | 25.3 | 4.97 | 83.4 | config | model | log |
| MogaNet-B | From scratch | DeiT | 224x224 | 43.9 | 9.93 | 84.3 | config | model | log |
| MogaNet-L | From scratch | DeiT | 224x224 | 82.5 | 15.9 | 84.7 | config | model | log |
| MogaNet-XL | From scratch | DeiT | 224x224 | 180.8 | 34.5 | 85.1 | config | model | log |
| MogaNet-XT | From scratch | RSB A3 | 160x160 | 2.97 | 0.80 | 72.8 | config | model | log |
| MogaNet-T | From scratch | RSB A3 | 160x160 | 5.20 | 1.10 | 75.4 | config | model | log |
| MogaNet-S | From scratch | RSB A3 | 160x160 | 25.3 | 4.97 | 81.1 | config | model | log |
| MogaNet-B | From scratch | RSB A3 | 160x160 | 43.9 | 9.93 | 82.2 | config | model | log |
| MogaNet-L | From scratch | RSB A3 | 160x160 | 43.9 | 9.93 | 83.2 | config | model | log |