Continuously tested on Linux, MacOS and Windows:
New 2021 paper:
Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g., human body pose estimation and tracking. In this work, we present a general framework that jointly detects and forms spatio-temporal keypoint associations in a single stage, making this the first real-time pose detection and tracking algorithm. We present a generic neural network architecture that uses Composite Fields to detect and construct a spatio-temporal pose which is a single, connected graph whose nodes are the semantic keypoints (e.g., a person's body joints) in multiple frames. For the temporal associations, we introduce the Temporal Composite Association Field (TCAF) which requires an extended network architecture and training method beyond previous Composite Fields. Our experiments show competitive accuracy while being an order of magnitude faster on multiple publicly available datasets such as COCO, CrowdPose and the PoseTrack 2017 and 2018 datasets. We also show that our method generalizes to any class of semantic keypoints such as car and animal parts to provide a holistic perception framework that is well suited for urban mobility such as self-driving cars and delivery robots.
Previous CVPR 2019 paper.
Detailed instructions on our OpenPifPaf Guide.
For developers, there is also the
which is the same guide but based on the latest code in the
Prediction of a network that was trained on the MS COCO dataset
pip3 install openpifpaf matplotlib python3 -m openpifpaf.predict docs/coco/000000081988.jpg --image-min-dpi=200 --show-file-extension=jpeg --image-output
Prediction of a network trained on the COCO WholeBody dataset
python -m openpifpaf.predict docs/wholebody/soccer.jpeg --checkpoint=shufflenetv2k30-wholebody --line-width=2 --decoder=cifcaf:0 --image-output
Prediction of a network that was trained on the ApolloCar3D dataset:
python -m openpifpaf.predict guide/images/peterbourg.jpg \ --checkpoint shufflenetv2k16-apollo-24 -o images \ --instance-threshold 0.05 --seed-threshold 0.05 \ --line-width 4 --font-size 0 --show-file-extension=jpeg
Prediction of a network that was trained on the AnimalPose dataset:
python -m openpifpaf.predict guide/images tappo_loomo.jpg \ --checkpoint=shufflenetv2k30-animalpose \ --line-width=6 --font-size=6 --white-overlay=0.3 \ --long-edge=500 --show-file-extension=jpeg