[Feature] Support Voxelpose #1050

wusize · 2021-12-03T12:48:49Z

Motivation

Support VoxelPose Algorithm in MMPose.

Modification

modify .dev_scripts/github/update_model_index.py to support multi-view 3d pose.
Add mmpose/core/camera/single_camera_torch.py to apply camera-transformation in pytorch.
Add affine_transform_torch in mmpose/core/post_processing/post_transforms.py.
Add MultiItemProcess and MultiItemDeduplicate in mmpose/datasets/pipelines/shared_transform.py
Delete unused variables (img) in mmpose/datasets/pipelines/bottom_up_transform.py.
Add config, dataset, models/detector, models/head, models/backbone and related docs.

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

CLA has been signed and all committers have signed the CLA in this PR.

codecov · 2021-12-03T15:26:48Z

Codecov Report

Attention: Patch coverage is 65.13761% with 304 lines in your changes missing coverage. Please review.

Please upload report for BASE (dev-0.22@4297d1e). Learn more about missing BASE report.

Files	Patch %	Lines
mmpose/models/detectors/voxelpose.py	34.13%	158 Missing and 6 partials ⚠️
...ets/body3d/body3d_mview_direct_panoptic_dataset.py	82.42%	29 Missing and 16 partials ⚠️
mmpose/datasets/pipelines/pose3d_transform.py	13.95%	37 Missing ⚠️
...tasets/base/kpt_3d_mview_rgb_img_direct_dataset.py	65.75%	24 Missing and 1 partial ⚠️
mmpose/core/camera/single_camera_torch.py	69.49%	14 Missing and 4 partials ⚠️
mmpose/datasets/pipelines/shared_transform.py	43.47%	13 Missing ⚠️
mmpose/models/heads/voxelpose_head.py	97.01%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             dev-0.22    #1050   +/-   ##
===========================================
  Coverage            ?   82.39%           
===========================================
  Files               ?      202           
  Lines               ?    16023           
  Branches            ?     2858           
===========================================
  Hits                ?    13202           
  Misses              ?     2096           
  Partials            ?      725

Flag	Coverage Δ
unittests	`82.35% <65.13%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

configs/body/3d_kpt_mview_rgb_img/voxelpose/panoptic/prn64x64x64_cpn80x80x20_panoptic_cam5.py

mmpose/core/camera/single_camera_torch.py

mmpose/datasets/datasets/base/kpt_3d_mview_rgb_img_direct_dataset.py

mmpose/datasets/pipelines/pose3d_transform.py

mmpose/datasets/pipelines/shared_transform.py

mmpose/models/detectors/voxelpose.py

mmpose/models/heads/voxelpose_head.py

* add the function to test the dummy forward speed of models. * add tools to test the flops and inference speed of multiple models.

* add configs * add dark configs * add checkpoint and readme * update webcam demo * fix model path in webcam demo * fix unittest

* add vipnas mbv3 * test other variants * submission for mmpose * add unittest * add readme * update .yml * fix lint * rebase * fix pytest Co-authored-by: jin-s13 <jinsheng13@foxmail.com>

…it_pose_model and fix some typo in cfg (#1028)

mmpose/models/heads/voxelpose_head.py

wusize · 2021-12-15T07:03:42Z

This pr could be merged.

* [Enhancement] inference speed and flops tools. (open-mmlab#986) * add the function to test the dummy forward speed of models. * add tools to test the flops and inference speed of multiple models. * [Feature] Add ViPNAS models for wholebody keypoint detection (open-mmlab#1009) * add configs * add dark configs * add checkpoint and readme * update webcam demo * fix model path in webcam demo * fix unittest * update model metafiles (open-mmlab#1001) * [Feature] Add ViPNAS mbv3 (open-mmlab#1025) * add vipnas mbv3 * test other variants * submission for mmpose * add unittest * add readme * update .yml * fix lint * rebase * fix pytest Co-authored-by: jin-s13 <jinsheng13@foxmail.com> * add cfg file for flops and speed test, change the bulid_posenet to init_pose_model and fix some typo in cfg (open-mmlab#1028) * Skip CI when some specific files were changed (open-mmlab#1041) * add voxelpose * unit test * unit test * unit test * add docs/ckpts * del unnecessary comments * correct typos in comments and docs * Add or modify docs * change variable names * reduce memory cost in test * get person_id * rebase * resolve comments * rebase master * rename cfg files * fix typos in comments Co-authored-by: zengwang430521 <zengwang430521@gmail.com> Co-authored-by: Yining Li <liyining0712@gmail.com> Co-authored-by: Lumin <30328525+luminxu@users.noreply.github.com> Co-authored-by: jin-s13 <jinsheng13@foxmail.com> Co-authored-by: Qikai Li <87690686+liqikai9@users.noreply.github.com> Co-authored-by: QwQ2000 <396707050@qq.com>

* [Enhancement] inference speed and flops tools. (#986) * add the function to test the dummy forward speed of models. * add tools to test the flops and inference speed of multiple models. * [Feature] Add ViPNAS models for wholebody keypoint detection (#1009) * add configs * add dark configs * add checkpoint and readme * update webcam demo * fix model path in webcam demo * fix unittest * update model metafiles (#1001) * [Feature] Add ViPNAS mbv3 (#1025) * add vipnas mbv3 * test other variants * submission for mmpose * add unittest * add readme * update .yml * fix lint * rebase * fix pytest Co-authored-by: jin-s13 <jinsheng13@foxmail.com> * add cfg file for flops and speed test, change the bulid_posenet to init_pose_model and fix some typo in cfg (#1028) * Skip CI when some specific files were changed (#1041) * add voxelpose * unit test * unit test * unit test * add docs/ckpts * del unnecessary comments * correct typos in comments and docs * Add or modify docs * change variable names * reduce memory cost in test * get person_id * rebase * resolve comments * rebase master * rename cfg files * fix typos in comments Co-authored-by: zengwang430521 <zengwang430521@gmail.com> Co-authored-by: Yining Li <liyining0712@gmail.com> Co-authored-by: Lumin <30328525+luminxu@users.noreply.github.com> Co-authored-by: jin-s13 <jinsheng13@foxmail.com> Co-authored-by: Qikai Li <87690686+liqikai9@users.noreply.github.com> Co-authored-by: QwQ2000 <396707050@qq.com>

* [Enhancement] inference speed and flops tools. (open-mmlab#986) * add the function to test the dummy forward speed of models. * add tools to test the flops and inference speed of multiple models. * [Feature] Add ViPNAS models for wholebody keypoint detection (open-mmlab#1009) * add configs * add dark configs * add checkpoint and readme * update webcam demo * fix model path in webcam demo * fix unittest * update model metafiles (open-mmlab#1001) * [Feature] Add ViPNAS mbv3 (open-mmlab#1025) * add vipnas mbv3 * test other variants * submission for mmpose * add unittest * add readme * update .yml * fix lint * rebase * fix pytest Co-authored-by: jin-s13 <jinsheng13@foxmail.com> * add cfg file for flops and speed test, change the bulid_posenet to init_pose_model and fix some typo in cfg (open-mmlab#1028) * Skip CI when some specific files were changed (open-mmlab#1041) * add voxelpose * unit test * unit test * unit test * add docs/ckpts * del unnecessary comments * correct typos in comments and docs * Add or modify docs * change variable names * reduce memory cost in test * get person_id * rebase * resolve comments * rebase master * rename cfg files * fix typos in comments Co-authored-by: zengwang430521 <zengwang430521@gmail.com> Co-authored-by: Yining Li <liyining0712@gmail.com> Co-authored-by: Lumin <30328525+luminxu@users.noreply.github.com> Co-authored-by: jin-s13 <jinsheng13@foxmail.com> Co-authored-by: Qikai Li <87690686+liqikai9@users.noreply.github.com> Co-authored-by: QwQ2000 <396707050@qq.com>

wusize requested a review from ly015 December 3, 2021 12:49

jin-s13 changed the title ~~Voxelpose~~ [Feature] Voxelpose Dec 4, 2021