[Feature] Support Concat dataset

[Canwang-sjtu]

Motivation

Fix #1106

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• CLA has been signed and all committers have signed the CLA in this PR.

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

Merging #1139 (e93e104) into dev-0.23 (d5fd7c6) will decrease coverage by 0.00%.
The diff coverage is 82.35%.

@@             Coverage Diff              @@
##           dev-0.23    #1139      +/-   ##
============================================
- Coverage     82.46%   82.46%   -0.01%     
============================================
  Files           203      203              
  Lines         16069    16101      +32     
  Branches       2862     2873      +11     
============================================
+ Hits          13252    13278      +26     
  Misses         2095     2095              
- Partials        722      728       +6     

Flag	Coverage Δ
unittests	82.41% <82.35%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmpose/datasets/builder.py	78.37% <82.35%> (+4.56%)	⬆️
mmpose/datasets/pipelines/shared_transform.py	81.14% <0.00%> (-0.44%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5fd7c6...e93e104. Read the comment docs.

[ly015]

Thank you very much for this PR. Would you be willing to add a unit test for the new feature?

[chenxinfeng4]

Error of 'ConcatDataset', saying 'ConcatDataset' object has no attribute 'evaluate'

I've update the code in 2022-2-22.

Traceback (most recent call last):
  File "tools/train.py", line 195, in <module>
    main()
  File "tools/train.py", line 191, in main
    meta=meta)
  File "/DATA/****/mmpose/mmpose/apis/train.py", line 191, in train_model
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/****/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/****/.conda/envs/mmpose/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
    self.call_hook('after_train_epoch')
  File "/home/****/.conda/envs/mmpose/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/****/.conda/envs/mmpose/lib/python3.7/site-packages/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch
    self._do_evaluate(runner)
  File "/home/****/.conda/envs/mmpose/lib/python3.7/site-packages/mmcv/runner/hooks/evaluation.py", line 505, in _do_evaluate
    key_score = self.evaluate(runner, results)
  File "****/mmpose/mmpose/core/evaluation/eval_hooks.py", line 135, in evaluate
    eval_res = self.dataloader.dataset.evaluate(
AttributeError: 'ConcatDataset' object has no attribute 'evaluate'

This is my config file

data = dict(
    workers_per_gpu=2,
    train_dataloader=dict(samples_per_gpu=24),
    val_dataloader=dict(samples_per_gpu=1),
    test_dataloader=dict(samples_per_gpu=1),
    train=dict(
        type = 'ConcatDataset',
        datasets = [dict(
            type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-02-bwrat_side6-kp_train.json',
            img_prefix='data/rat_keypoints/2021-11-02-bwrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=train_pipeline,
            dataset_info=dataset_info),
        dict(
            type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-24-wrat_side6-kp_train.json',
            img_prefix='data/rat_keypoints/2021-11-24-wrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=train_pipeline,
            dataset_info=dataset_info)
        ]),
    val=dict(
        type = 'ConcatDataset',
        datasets = [dict(type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-02-bwrat_side6-kp_val.json',
            img_prefix='data/rat_keypoints/2021-11-02-bwrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=val_pipeline,
            dataset_info=dataset_info),
        dict(
            type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-24-wrat_side6-kp_val.json',
            img_prefix='data/rat_keypoints/2021-11-24-wrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=val_pipeline,
            dataset_info=dataset_info)
        ]),
    test=dict(
        type = 'ConcatDataset',
        datasets = [dict(
            type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-02-bwrat_side6-kp_val.json',
            img_prefix='data/rat_keypoints/2021-11-02-bwrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=test_pipeline,
            dataset_info=dataset_info),
        dict(
            type='BottomUpCocoDataset',
            ann_file='data/rat_keypoints/2021-11-24-wrat_side6-kp_val.json',
            img_prefix='data/rat_keypoints/2021-11-24-wrat_side6-kp/',
            data_cfg=data_cfg,
            pipeline=val_pipeline,
            dataset_info=dataset_info)
        ])
)