DecenterNet: Delving into Multi-person Pose Estimation in Crowded Scenes

1. Introduction

This is the official repository for our paper: DecenterNet: Delving into Multi-person Pose Estimation in Crowded Scenes, which is accepted by ACM MM 2023 (the 31st ACM International Conference on Multimedia). This paper presents DecenterNet for multi-person pose estimation in crowded scenes. It represents the human pose with a decentralized pose representation that applies all visible keypoints as the root points, making it more robust in the crowd. It also introduces a location map to model the point selection function of the heatmap and enhance its scoring function. To better evaluate model performance in entangled scenes, we construct the SkatingPose dataset and design an Entanglement Index to measure the entangled level of an image. DecenterNet significantly outperforms state-of-the-art methods on the SkatingPose, CrowdPose, and COCO datasets.

2. Enviroment

1. Clone this repo, and we'll call the directory that you cloned as ${POSE_ROOT}.

2. Install dependencies:

   pip install -r requirements.txt
   

3. Install COCOAPI:

   # COCOAPI=/path/to/clone/cocoapi
   git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
   cd $COCOAPI/PythonAPI
   # Install into global site-packages
   make install
   

4. Install CrowdPoseAPI exactly the same as COCOAPI.

3. Data preparation

For COCO data, please download from COCO download, 2017 Train/Val is needed for COCO keypoints training and validation. Download and extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- coco
    `-- |-- annotations
        |   |-- person_keypoints_train2017.json
        |   `-- person_keypoints_val2017.json
        `-- images
            |-- train2017.zip
            `-- val2017.zip

For CrowdPose data, please download from CrowdPose download, Train/Val is needed for CrowdPose keypoints training. Download and extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- crowdpose
    `-- |-- json
        |   |-- crowdpose_train.json
        |   |-- crowdpose_val.json
        |   |-- crowdpose_trainval.json (generated by tools/crowdpose_concat_train_val.py)
        |   `-- crowdpose_test.json
        `-- images.zip

After downloading data, run python tools/crowdpose_concat_train_val.py under ${POSE_ROOT} to create trainval set.

Model Zoo:

Google Drive make them look like this:

${POSE_ROOT}
|-- data
|-- models
`-- |-- imagenet_backbone
    `-- pretrained_models
...

Training and Testing

Testing on COCO val2017 dataset

python tools/launch.py  --cfg experiments/coco/w32/w32_4x_reg03_bs10_512_adam_lr1e-3_coco_x140.yaml --versionid 0 --train 0 --model_path 'models/pretrained_models/coco_w32/epoch=129-step=74490.ckpt'

Testing on CrowdPose test set

python tools/launch.py  --cfg experiments/crowdpose/w32/w32_4x_reg03_bs10_512_adam_lr1e-3_crowdpose_x300.yaml --versionid 0 --train 0 --model_path 'models/pretrained_models/crowd_w32/epoch=269-step=29160.ckpt'

Training on COCO val2017 dataset

python tools/launch.py  --cfg experiments/coco/w32/w32_4x_reg03_bs10_512_adam_lr1e-3_coco_x140.yaml --versionid 0 --train 1

Other content will come soon...

The repository is still under construction. Please feel free to ask if you encounter any issues.