[Feature] Visualize openpose skeleton style on the image #2115
Conversation
|
Use 'mim install "mmpose>=1.0.0rc1"' to install mmpose, and an error will be reported when running the code. But using "pip install -v -e ." to install mmpose, running the code doesn't throw any error. |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## dev-1.x #2115 +/- ##
===========================================
- Coverage 82.42% 82.30% -0.12%
===========================================
Files 227 227
Lines 13324 13340 +16
Branches 2257 2261 +4
===========================================
- Hits 10982 10980 -2
- Misses 1834 1845 +11
- Partials 508 515 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
Thank you for your contribution! There's no need to fret about this issue. The recently introduced features won't be incorporated into mmpose when installed via pip or mim until a new release is available. However, these features are indeed present in the source code, so they will be accessible if mmpose is installed directly from the source. |
OK，thank you |
| @@ -243,6 +248,20 @@ def _draw_instances_kpts(self, | |||
| else: | |||
| keypoints_visible = [np.ones(len(kpts)) for kpts in keypoints] | |||
|
|
|||
| if skeleton_style == 'openpose': | |||
| neck = (keypoints[:, 5] + keypoints[:, 6]) / 2 | |||
| neck_score = (scores[:, 5] + scores[:, 6]) / 2 | |||
There was a problem hiding this comment.
Regarding the confidence score for the neck, I think it should be considered based on different situations:
- When both
left_shoulderandright_shoulderare present (score >= args.kpt_thr), the average score of the two can be used as the score for the neck. - When one shoulder is missing, the score for the neck should be 0
There was a problem hiding this comment.
BTW, can you also fix this in projects/mmpose4aigc/openpose_visualization.py this line?
| new_cols = [1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17] | ||
| keypoints[:, new_cols] = keypoints[:, original_cols] | ||
| scores[:, new_cols] = scores[:, original_cols] | ||
| keypoints_visible = [np.ones(len(kpts)) for kpts in keypoints] |
There was a problem hiding this comment.
When visualizing images from GT in openpose-style, keypoints_visible should not be np.ones() for all keypoints
There was a problem hiding this comment.
Does keypoints_visible to the joint_weights in coco_openpose.py?Here I refer to https://github.com/open-mmlab/mmpose/pull/2115/files/9630c10a895bec012cd44b2a882f8c1c143d9318#diff-070475f91f5ca244130def9db60fe354566df801e1a23aa33d3b2060c9c8515cL244,I don't understand keypoints_visible
There was a problem hiding this comment.
just refer to L246-L249:
if 'keypoints_visible' in instances:
keypoints_visible = instances.keypoints_visible
else:
keypoints_visible = [np.ones(len(kpts)) for kpts in keypoints]joint_weights is used in loss calculation.
There was a problem hiding this comment.
When visualizing images from GT in openpose-style,
keypoints_visibleshould not be np.ones() for all keypoints
Thank you very much for your explanation, I understand now!
demo/image_demo.py
Outdated
| if dataset_meta.get( | ||
| 'dataset_name') == 'coco' and args.skeleton_style == 'openpose': | ||
| dataset_meta = parse_pose_metainfo( | ||
| dict(from_file='configs/_base_/datasets/coco_openpose.py')) |
There was a problem hiding this comment.
Should we move this piece of code into the visualizer?
There was a problem hiding this comment.
ok，Let me try
Motivation
Modification
BC-breaking (Optional)
Use cases (Optional)
add
--skeleton-styletoimage_demo.pyandtopdown_demo_with_mmdet.pyFor Example:
python demo/topdown_demo_with_mmdet.py \ ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \ ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \ --input ${INPUT_PATH} \ --skeleton-style openpose \ --showThe result is as follow:
Checklist
Before PR:
After PR: