UniPose

Download Pretained Model

See Releases

RUN on PC:

python unipose.py --pretrained pretrained/UniPose_MPII.pth

Run on 200dk

python run_unipose.py

• Performace on 200dk

  Model	Pre-process	Inference	Post-process	Sum	FPS
  Unipose	0.020776	0.017968	0.115213	0.153957	6.495
  Unipose merge post-process by nonzero op	0.022704	0.968188	0	0.990892	1.009
  Unipose merge post-process by argmax op	0.01951217	0.1023523	0	0.121864	8.206

---

UniPose: Unified Human Pose Estimation in Single Images and Videos.

Figure 1: UniPose architecture for single frame pose detection. The input color image of dimensions (HxW) is fed through the ResNet backbone and WASP module to obtain 256 feature channels at reduced resolution by a factor of 8. The decoder module generates K heatmaps, one per joint, at the original resolution, and the locations of the joints are determined by a local max operation.

Figure 2: UniPose-LSTM architecture for pose estimation in videos. The joint heatmaps from the decoder of UniPose are fed into the LSTM along with the final heatmaps from the previous LSTM state. The convolutional layers following the LSTM reorganize the outputs into the final heatmaps used for joint localization.

We propose UniPose, a unified framework for human pose estimation, based on our "Waterfall" Atrous Spatial Pooling architecture, that achieves state-of-art-results on several pose estimation metrics. UniPose incorporates contextual segmentation and joint localization to estimate the human pose in a single stage, with high accuracy, without relying on statistical postprocessing methods. The Waterfall module in UniPose leverages the efficiency of progressive filtering in the cascade architecture, while maintaining multi-scale fields-of-view comparable to spatial pyramid configurations. Additionally, our method is extended to UniPose-LSTM for multi-frame processing and achieves state-of-the-art results for temporal pose estimation in Video. Our results on multiple datasets demonstrate that UniPose, with a ResNet backbone and Waterfall module, is a robust and efficient architecture for pose estimation obtaining state-of-the-art results in single person pose detection for both single images and videos.

We propose the “Waterfall Atrous Spatial Pyramid” module, shown in Figure 3. WASP is a novel architecture with Atrous Convolutions that is able to leverage both the larger Field-of-View of the Atrous Spatial Pyramid Pooling configuration and the reduced size of the cascade approach.

Figure 3: WASP Module.

Examples of the UniPose architecture for Pose Estimation are shown in Figures 4 for single images and videos.

Figure 4: Pose estimation samples for UniPose in images and videos.

Link to the published article at CVPR 2020.

Datasets:

Datasets used in this paper and required for training, validation, and testing can be downloaded directly from the dataset websites below:
LSP Dataset: https://sam.johnson.io/research/lsp.html
MPII Dataset: http://human-pose.mpi-inf.mpg.de/
PennAction Dataset: http://dreamdragon.github.io/PennAction/
BBC Pose Dataset: https://www.robots.ox.ac.uk/~vgg/data/pose/

Pre-trained Models:

The pre-trained weights can be downloaded here.

NEW!: UniPose+: A unified framework for 2D and 3D human pose estimation in images and videos

Our novel and improved UniPose+ framework for pose estimation achieves State-of-the-Art results in several datasets. UniPose+ can be found in the following link: UniPose+ at PAMI. Full code for the UniPose+ framework is scheduled to be released in the near future.

NEW!: OmniPose: A Multi-Scale Framework for Multi-Person Pose Estimation

Our novel framework for multi-person pose estimation achieves State-of-the-Art results in several datasets. The pre-print of our new method, OmniPose, can be found in the following link: OmniPose pre-print. Full code for the OmniPose framework is scheduled to be released in the near future.

Contact:

Bruno Artacho:
E-mail: bmartacho@mail.rit.edu
Website: https://www.brunoartacho.com

Andreas Savakis:
E-mail: andreas.savakis@rit.edu
Website: https://www.rit.edu/directory/axseec-andreas-savakis

Citation:

Artacho, B.; Savakis, A. UniPose: Unified Human Pose Estimation in Single Images and Videos. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

Latex:
@InProceedings{Artacho_2020_CVPR,
title = {UniPose: Unified Human Pose Estimation in Single Images and Videos},
author = {Artacho, Bruno and Savakis, Andreas},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020},
}