PosePlusSeg (AAAI 2022)

Official code repo for the paper "Joint Human Pose Estimation and Instance Segmentation with PosePlusSeg"[arXiv] version will be available soon.

Model Architecture

Setup environment

- python==3.6
- conda install -c conda-forge matplotlib==2.0.2
- conda install -c conda-forge opencv OR pip install opencv-python
- conda install -c conda-forge pycocotools
- conda install -c anaconda scikit-image
- conda install tensorflow-gpu==1.13.1

Recomendation: tensorflow 1.13 & coda 10.

Download data

COCO 2017

• COCO 2017 Train images 118K/18GB
• COCO 2017 Val images 5K/1GB
• COCOPersons Train Annotation (person_keypoints_train2017_pose2seg.json) [166MB]
• COCOPersons Val Annotation (person_keypoints_val2017_pose2seg.json) [7MB]

Hint

Person keypoint dataset is a subset of COCO2017 dataset (COCO 2017 Train images 118K/18GB). We train our model only on human instances key points and segmentation by introducing a multi task system.

Setup data

The coco2017 folder should be like this:

├── coco2017
│   ├── annotations  
│   │   ├── person_keypoints_train2017.json 
│   │   ├── person_keypoints_val2017.json 
│   ├── train2017  
│   │   ├── ####.jpg  
│   ├── val2017  
│   │   ├── ####.jpg  

Train

Run the python train.py for training the model.

Note

1. Please correctly give the path to the dataset folder and check point files in the config.py file.
2. Currently we only support single-gpu training (Recommended: TITAN RTX).

Test

Please lookout the PosePlusSeg_Test folder for testing the model.

Visualize Results

Human Pose Estimation

Human Instance Segmentation

Joint Human Pose Estimation and Instance Segmentation