[Feature] Support RLE for COCO

[Indigo6]

Motivation

Support RLE for COCO and add pre-trained models & log to the model zoo

Arch	Input Size	AP	Total Epochs	Gpu Num
deeppose_resnet_50_rle	256x192	69.35	210	2

Log and pre-trained model: Google Drive

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• CLA has been signed and all committers have signed the CLA in this PR.

[codecov]

Codecov Report

Merging #1424 (d240b5d) into dev-0.28 (fef003a) will increase coverage by 0.09%.
The diff coverage is 95.45%.

@@             Coverage Diff              @@
##           dev-0.28    #1424      +/-   ##
============================================
+ Coverage     84.35%   84.44%   +0.09%     
============================================
  Files           232      233       +1     
  Lines         19279    19402     +123     
  Branches       3468     3504      +36     
============================================
+ Hits          16262    16384     +122     
+ Misses         2152     2151       -1     
- Partials        865      867       +2     

Flag	Coverage Δ
unittests	84.37% <95.45%> (+0.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...datasets/datasets/top_down/topdown_coco_dataset.py	89.74% <92.30%> (+0.16%)	⬆️
mmpose/models/detectors/top_down.py	76.61% <100.00%> (+0.57%)	⬆️
mmpose/models/heads/deeppose_regression_head.py	96.55% <100.00%> (+3.86%)	⬆️
mmpose/core/optimizer/builder.py
mmpose/core/optimizer/__init__.py
mmpose/core/optimizers/builder.py	100.00% <0.00%> (ø)
mmpose/core/optimizers/__init__.py	100.00% <0.00%> (ø)
...re/optimizers/layer_decay_optimizer_constructor.py	96.11% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fef003a...d240b5d. Read the comment docs.

[ly015]

@Indigo6 Thank you very much! We will add other experiment settings (res101, res152) in this PR.

[Indigo6]

@Indigo6 Thank you very much! We will add other experiment settings (res101, res152) in this PR.

You're welcome. I may also open other PRs for coord regression with HRNet, Uncertainty L1/L2 loss

[Indigo6]

I finished experiments on res101_256x192 but got a result worse than res50.
I look forward to your experimental results and discussion, since there are no official results on res101_256x192.

[Ben-Louis]

I finished experiments on res101_256x192 but got a result worse than res50. I look forward to your experimental results and discussion, since there are no official results on res101_256x192.

Here are the test results of models trained with 8 GPUs

Arch	Input Size	AP	AP50	AP75
deeppose_resnet_50_rle	256x192	69.3	87.9	76.1
deeppose_resnet_101_rle	256x192	71.2	88.8	78.2
deeppose_resnet_152_rle	256x192	72.0	88.9	79.2

In our experiments, resnet101 performs better than resnet50.

But the AP of resnet50 and resnet152 are worse than that in the paper. We are searching for a better setting for RLE loss on COCO.

[Indigo6]

But the AP of resnet50 and resnet152 are worse than that in the paper. We are searching for a better setting for RLE loss on COCO.

Thanks a lot for sharing these! May I ask for your settings and logs?

[Ben-Louis]

But the AP of resnet50 and resnet152 are worse than that in the paper. We are searching for a better setting for RLE loss on COCO.

Thanks a lot for sharing these! May I ask for your settings and logs?

For the three models mentioned above, we use the config you provided. The config files and logs are uploaded to Google Drive. Further results will also be saved in this folder.

[Indigo6]

But the AP of resnet50 and resnet152 are worse than that in the paper. We are searching for a better setting for RLE loss on COCO.

Thanks a lot for sharing these! May I ask for your settings and logs?

For the three models mentioned above, we use the config you provided. The config files and logs are uploaded to Google Drive. Further results will also be saved in this folder.

Thanks a lot! I'll try again on my devices.
BTW, the reason the APs are worse than those in the paper may be with/without backbone weights pretrained with heatmap loss.

[Ben-Louis]

Thanks a lot! I'll try again on my devices. BTW, the reason the APs are worse than those in the paper may be with/without backbone weights pretrained with heatmap loss.

In the paper, even without pertaining with heatmap loss, the APs of r50/r152 are 70.5/74.2, respectively. They are higher than our results (69.3/72.0).

The official implementation uses different hyper-parameters from default setting of deeppose in MMPose, which might be the cause of different performance.

[Indigo6]

The official implementation uses different hyper-parameters from default setting of deeppose in MMPose, which might be the cause of different performance.

You're right, especially end epoch 210/270.

[Ben-Louis]

In our experiments, model gains neglectable performance improvement after epoch 210.

Arch	Input Size	lr	samples per gpu	data aug	total epoch	AP	AP50	AP75
deeppose_resnet_50_rle	256x192	5r-4	64	MMPose	210	69.3	87.9	76.1
deeppose_resnet_50_rle	256x192	1e-3	64	MMPose	210	69.5	87.9	76.7
deeppose_resnet_50_rle	256x192	1e-3	64	MMPose	270	69.6	88.0	76.7
deeppose_resnet_50_rle	256x192	1e-3	32	Official	270	69.1	87.9	76.1

We also tried the setting used in official implementation (batch size, data aug). We found that the loss is lower in training, due to simpler data aug. However the performance reduces with this setting.

[Indigo6]

I finished experiments on res101_256x192 but got a result worse than res50. I look forward to your experimental results and discussion, since there are no official results on res101_256x192.

Two experimental results on my devices

lr	samples per gpu	gpu num	warmup	total epoch	AP	log
1e-3	64	2	no	210	66.91	Google Drive
5e-4	64	2	yes	210	71.15	Google Drive

[ly015]

This markdown file can not be rendered. Could you please check if there is something wrong with the format? @Ben-Louis

[Ben-Louis]

Sure, it has been fixed.