Refactor MMClsWandbHook (#1)

* [Enhance] Add extra dataloader settings in configs. (open-mmlab#752)

* Use `train_dataloader`, `val_dataloader` and `test_dataloader` settings
in the `data` field to specify different arguments.

* Fix bug

* Fix bug

* [Enhance] Improve CPE performance by reduce memory copy. (open-mmlab#762)

* [Feature] Support resize relative position embedding in `SwinTransformer`. (open-mmlab#749)

* [Feature]: Add resize rel pos embed

* [Refactor]: Create a separated resize_rel_pos_bias_table func

* [Refactor]: Refactor rel pos embed bias

* [Refactor]: Move interpolate into func

* Remove index buffer only when window_size changes

Co-authored-by: mzr1996 <mzr1996@163.com>

* [Feature] Add PoolFormer backbone and checkpoints. (open-mmlab#746)

* add PoolFormer

* fix some typos in PoolFormer

* fix lint error

* modify out_indices and gap

* fix typo

* fix lint

* fix typo

* fix typo in poolforemr README

* fix lint

* Update some paths

* Refactor freeze_stages method

* Add unit tests

* Fix lint

Co-authored-by: mzr1996 <mzr1996@163.com>

* Bump version to v0.22.1 (open-mmlab#785)

* [Docs] Refine API reference. (open-mmlab#774)

* [Docs] Refine API reference

* Add PoolFormer

* [Docs] Fix docs.

* [Enhance] Reduce the memory usage of unit tests for Swin-Transformer. (open-mmlab#759)

* [Feature] Support VAN. (open-mmlab#739)

* add van

* fix config

* add metafile

* add test

* model convert script

* fix review

* fix lint

* fix the configs and improve docs

* rm debug lines

* add VAN into api

Co-authored-by: Yu Zhaohui <1105212286@qq.com>

* [Feature] Support DenseNet. (open-mmlab#750)

* init add densenet implementation

* Add config and converted models

* update meta

* add test for memory efficient

* Add docs

* add doc for jit

* Update checkpoint path

* Update readthedocs

Co-authored-by: mzr1996 <mzr1996@163.com>

* [Fix] Use symbolic link in the API reference of Chinese docs.

* [Enhance] Support training on IPU and add fine-tuning configs of ViT. (open-mmlab#723)

* implement training and evaluation on IPU

* fp16 SOTA

* Tput reaches 5600

* 123

* add poptorch dataloder

* change ipu_replicas to ipu-replicas

* add noqa to config long line(website)

* remove ipu dataloder test code

* del one blank line in test_builder

* refine the dataloder initialization

* fix a typo

* refine args for dataloder

* remove an annoted line

* process one more conflict

* adjust code structure in mmcv.ipu

* adjust ipu code structure in mmcv

* IPUDataloader to IPUDataLoader

* align with mmcv

* adjust according to mmcv

* mmcv code structre fixed

Co-authored-by: hudi <dihu@graphcore.ai>

* [Fix] Fix lint and mmcv version requirement for IPU.

* Bump version to v0.23.0 (open-mmlab#809)

* Refacoter Wandb hook and refine docstring

Co-authored-by: XiaobingZhang <xiaobing.zhang@intel.com>
Co-authored-by: Yuan Liu <30762564+YuanLiuuuuuu@users.noreply.github.com>
Co-authored-by: Weihao Yu <1090924009@qq.com>
Co-authored-by: takuoko <to78314910@gmail.com>
Co-authored-by: Yu Zhaohui <1105212286@qq.com>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
Co-authored-by: Hu Di <476658825@qq.com>
Co-authored-by: hudi <dihu@graphcore.ai>

.gitignore

@@ -69,10 +69,12 @@ docs/en/_build/
 docs/en/_model_zoo.rst
 docs/en/modelzoo_statistics.md
 docs/en/papers/
+docs/en/api/generated/
 docs/zh_CN/_build/
 docs/zh_CN/_model_zoo.rst
 docs/zh_CN/modelzoo_statistics.md
 docs/zh_CN/papers/
+docs/zh_CN/api/generated/
 
 # PyBuilder
 target/
@@ -120,6 +122,13 @@ venv.bak/
 *.log.json
 /work_dirs
 /mmcls/.mim
+.DS_Store
 
 # Pytorch
 *.pth
+
+# IPU
+*.pvti
+*.pvti-journal
+/cache_engine
+/report

README.md

@@ -59,20 +59,19 @@ The master branch works with **PyTorch 1.5+**.
 
 ## What's new
 
+v0.23.0 was released in 1/5/2022.
+Highlights of the new version:
+- Support **DenseNet**, **VAN** and **PoolFormer**, and provide pre-trained models.
+- Support training on IPU.
+- New style API docs, welcome [view it](https://mmclassification.readthedocs.io/en/master/api/models.html).
+
 v0.22.0 was released in 30/3/2022.
 
 Highlights of the new version:
 - Support a series of **CSP Network**, such as CSP-ResNet, CSP-ResNeXt and CSP-DarkNet.
 - A new `CustomDataset` class to help you **build dataset of yourself**!
 - Support new backbones - **ConvMixer**, **RepMLP** and new dataset - **CUB dataset**.
 
-v0.21.0 was released in 4/3/2022.
-
-Highlights of the new version:
-- Support **ResNetV1c** and **Wide-ResNet**, and provide pre-trained models.
-- Support **dynamic input shape** for ViT-based algorithms. Now our ViT, DeiT, Swin-Transformer and T2T-ViT support forwarding with any input shape.
-- Reproduce training results of DeiT. And our DeiT-T and DeiT-S have **higher accuracy** comparing with the official weights.
-
 Please refer to [changelog.md](docs/en/changelog.md) for more details and other release history.
 
 ## Installation
@@ -138,8 +137,10 @@ Results and models are available in the [model zoo](https://mmclassification.rea
 - [x] [EfficientNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/efficientnet)
 - [x] [ConvNeXt](https://github.com/open-mmlab/mmclassification/tree/master/configs/convnext)
 - [x] [HRNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/hrnet)
+- [x] [VAN](https://github.com/open-mmlab/mmclassification/tree/master/configs/van)
 - [x] [ConvMixer](https://github.com/open-mmlab/mmclassification/tree/master/configs/convmixer)
 - [x] [CSPNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/cspnet)
+- [x] [PoolFormer](https://github.com/open-mmlab/mmclassification/tree/master/configs/poolformer)
 
 </details>
 

README_zh-CN.md

@@ -57,20 +57,20 @@ MMClassification 是一款基于 PyTorch 的开源图像分类工具箱，是 [O
 
 ## 更新日志
 
+2022/5/1 发布了 v0.23.0 版本
+
+新版本亮点：
+- 支持了 **DenseNet**，**VAN** 和 **PoolFormer** 三个网络，并提供了预训练模型。
+- 支持在 IPU 上进行训练。
+- 更新了 API 文档的样式，更方便查阅，[欢迎查阅](https://mmclassification.readthedocs.io/en/master/api/models.html)。
+
 2022/3/30 发布了 v0.22.0 版本
 
 新版本亮点：
 - 支持了一系列 **CSP Net**，包括 CSP-ResNet，CSP-ResNeXt 和 CSP-DarkNet。
 - 我们提供了一个新的 `CustomDataset` 类，这个类将帮助你轻松使用**自己的数据集**！
 - 支持了新的主干网络 **ConvMixer**、**RepMLP** 和一个新的数据集 **CUB dataset**。
 
-2022/3/4 发布了 v0.21.0 版本
-
-新版本亮点：
-- 支持了 **ResNetV1c** 和 **Wide-ResNet** 两个 ResNet 变种，并提供了预训练模型
-- ViT 相关模型支持 **动态输入尺寸**。现在我们的 ViT，DeiT，Swin-Transformer 和 T2T-ViT 支持任意尺寸的输入。
-- 复现了 DeiT 的训练结果，并且我们的 DeiT-T 和 DeiT-S 拥有比官方权重 **更高的精度**。
-
 发布历史和更新细节请参考 [更新日志](docs/en/changelog.md)
 
 ## 安装
@@ -136,8 +136,10 @@ pip3 install -e .
 - [x] [EfficientNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/efficientnet)
 - [x] [ConvNeXt](https://github.com/open-mmlab/mmclassification/tree/master/configs/convnext)
 - [x] [HRNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/hrnet)
+- [x] [VAN](https://github.com/open-mmlab/mmclassification/tree/master/configs/van)
 - [x] [ConvMixer](https://github.com/open-mmlab/mmclassification/tree/master/configs/convmixer)
 - [x] [CSPNet](https://github.com/open-mmlab/mmclassification/tree/master/configs/cspnet)
+- [x] [PoolFormer](https://github.com/open-mmlab/mmclassification/tree/master/configs/poolformer)
 
 </details>
 

configs/_base_/datasets/imagenet_bs128_poolformer_medium_224.py

@@ -0,0 +1,71 @@
+_base_ = ['./pipelines/rand_aug.py']
+
+# dataset settings
+dataset_type = 'ImageNet'
+img_norm_cfg = dict(
+    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+
+train_pipeline = [
+    dict(type='LoadImageFromFile'),
+    dict(
+        type='RandomResizedCrop',
+        size=224,
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
+    dict(
+        type='RandAugment',
+        policies={{_base_.rand_increasing_policies}},
+        num_policies=2,
+        total_level=10,
+        magnitude_level=9,
+        magnitude_std=0.5,
+        hparams=dict(
+            pad_val=[round(x) for x in img_norm_cfg['mean'][::-1]],
+            interpolation='bicubic')),
+    dict(
+        type='RandomErasing',
+        erase_prob=0.25,
+        mode='rand',
+        min_area_ratio=0.02,
+        max_area_ratio=1 / 3,
+        fill_color=img_norm_cfg['mean'][::-1],
+        fill_std=img_norm_cfg['std'][::-1]),
+    dict(type='Normalize', **img_norm_cfg),
+    dict(type='ImageToTensor', keys=['img']),
+    dict(type='ToTensor', keys=['gt_label']),
+    dict(type='Collect', keys=['img', 'gt_label'])
+]
+
+test_pipeline = [
+    dict(type='LoadImageFromFile'),
+    dict(
+        type='Resize',
+        size=(236, -1),
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='CenterCrop', crop_size=224),
+    dict(type='Normalize', **img_norm_cfg),
+    dict(type='ImageToTensor', keys=['img']),
+    dict(type='Collect', keys=['img'])
+]
+data = dict(
+    samples_per_gpu=128,
+    workers_per_gpu=8,
+    train=dict(
+        type=dataset_type,
+        data_prefix='data/imagenet/train',
+        pipeline=train_pipeline),
+    val=dict(
+        type=dataset_type,
+        data_prefix='data/imagenet/val',
+        ann_file='data/imagenet/meta/val.txt',
+        pipeline=test_pipeline),
+    test=dict(
+        # replace `data/val` with `data/test` for standard test
+        type=dataset_type,
+        data_prefix='data/imagenet/val',
+        ann_file='data/imagenet/meta/val.txt',
+        pipeline=test_pipeline))
+
+evaluation = dict(interval=10, metric='accuracy')

configs/_base_/datasets/imagenet_bs128_poolformer_small_224.py

@@ -0,0 +1,71 @@
+_base_ = ['./pipelines/rand_aug.py']
+
+# dataset settings
+dataset_type = 'ImageNet'
+img_norm_cfg = dict(
+    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+
+train_pipeline = [
+    dict(type='LoadImageFromFile'),
+    dict(
+        type='RandomResizedCrop',
+        size=224,
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
+    dict(
+        type='RandAugment',
+        policies={{_base_.rand_increasing_policies}},
+        num_policies=2,
+        total_level=10,
+        magnitude_level=9,
+        magnitude_std=0.5,
+        hparams=dict(
+            pad_val=[round(x) for x in img_norm_cfg['mean'][::-1]],
+            interpolation='bicubic')),
+    dict(
+        type='RandomErasing',
+        erase_prob=0.25,
+        mode='rand',
+        min_area_ratio=0.02,
+        max_area_ratio=1 / 3,
+        fill_color=img_norm_cfg['mean'][::-1],
+        fill_std=img_norm_cfg['std'][::-1]),
+    dict(type='Normalize', **img_norm_cfg),
+    dict(type='ImageToTensor', keys=['img']),
+    dict(type='ToTensor', keys=['gt_label']),
+    dict(type='Collect', keys=['img', 'gt_label'])
+]
+
+test_pipeline = [
+    dict(type='LoadImageFromFile'),
+    dict(
+        type='Resize',
+        size=(248, -1),
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='CenterCrop', crop_size=224),
+    dict(type='Normalize', **img_norm_cfg),
+    dict(type='ImageToTensor', keys=['img']),
+    dict(type='Collect', keys=['img'])
+]
+data = dict(
+    samples_per_gpu=128,
+    workers_per_gpu=8,
+    train=dict(
+        type=dataset_type,
+        data_prefix='data/imagenet/train',
+        pipeline=train_pipeline),
+    val=dict(
+        type=dataset_type,
+        data_prefix='data/imagenet/val',
+        ann_file='data/imagenet/meta/val.txt',
+        pipeline=test_pipeline),
+    test=dict(
+        # replace `data/val` with `data/test` for standard test
+        type=dataset_type,
+        data_prefix='data/imagenet/val',
+        ann_file='data/imagenet/meta/val.txt',
+        pipeline=test_pipeline))
+
+evaluation = dict(interval=10, metric='accuracy')

configs/_base_/models/densenet/densenet121.py

@@ -0,0 +1,11 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(type='DenseNet', arch='121'),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=1024,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/densenet/densenet161.py

@@ -0,0 +1,11 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(type='DenseNet', arch='161'),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=2208,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/densenet/densenet169.py

@@ -0,0 +1,11 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(type='DenseNet', arch='169'),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=1664,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/densenet/densenet201.py

@@ -0,0 +1,11 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(type='DenseNet', arch='201'),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=1920,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/poolformer/poolformer_m36.py

@@ -0,0 +1,22 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(
+        type='PoolFormer',
+        arch='m36',
+        drop_path_rate=0.1,
+        init_cfg=[
+            dict(
+                type='TruncNormal',
+                layer=['Conv2d', 'Linear'],
+                std=.02,
+                bias=0.),
+            dict(type='Constant', layer=['GroupNorm'], val=1., bias=0.),
+        ]),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=768,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/poolformer/poolformer_m48.py

@@ -0,0 +1,22 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(
+        type='PoolFormer',
+        arch='m48',
+        drop_path_rate=0.1,
+        init_cfg=[
+            dict(
+                type='TruncNormal',
+                layer=['Conv2d', 'Linear'],
+                std=.02,
+                bias=0.),
+            dict(type='Constant', layer=['GroupNorm'], val=1., bias=0.),
+        ]),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=768,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/poolformer/poolformer_s12.py

@@ -0,0 +1,22 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(
+        type='PoolFormer',
+        arch='s12',
+        drop_path_rate=0.1,
+        init_cfg=[
+            dict(
+                type='TruncNormal',
+                layer=['Conv2d', 'Linear'],
+                std=.02,
+                bias=0.),
+            dict(type='Constant', layer=['GroupNorm'], val=1., bias=0.),
+        ]),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=512,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))

configs/_base_/models/poolformer/poolformer_s24.py

@@ -0,0 +1,22 @@
+# Model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(
+        type='PoolFormer',
+        arch='s24',
+        drop_path_rate=0.1,
+        init_cfg=[
+            dict(
+                type='TruncNormal',
+                layer=['Conv2d', 'Linear'],
+                std=.02,
+                bias=0.),
+            dict(type='Constant', layer=['GroupNorm'], val=1., bias=0.),
+        ]),
+    neck=dict(type='GlobalAveragePooling'),
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=512,
+        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
+    ))