PartHOE

Since I graduated from school in 2024.07, the environment has not remained the original version, and the code might have some bugs ~

If you need any help, feel free to ask me through GitHub issues!

News

⭐ Paper is release on arxiv PartHOE.

🎉 The paper is accepted by IROS 2024!

Environment Setup

conda create -n part_hoe python=3.7
conda activate part_hoe
pip install -r requirement.txt
pip install timm==0.4.9 einops

Dataset Preparation

Install COCOAPI

# COCOAPI=/path/to/clone/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI
# Install into global site-packages
make install
# Alternatively, if you do not have permissions or prefer
# not to install the COCO API into global site-packages
python3 setup.py install --user

Prepare the Dataset, whole-body joint annotations, and orientation annotations

Download 4 annotations and put them into the COCO annotation folder.

|-- data
`-- |-- coco
    `-- |-- annotations
        |   |-- coco_wholebody_train_v1.0.json
        |   |-- coco_wholebody_val_v1.0.json
        |   |-- merged_orientation_train.json
        |   |-- merged_orientation_val.json
        `-- images
            |-- train2017
            |   |-- 000000000009.jpg
            |   |-- 000000000025.jpg
            |   |-- 000000000030.jpg
            |   |-- ... 
            `-- val2017
                |-- 000000000139.jpg
                |-- 000000000285.jpg
                |-- 000000000632.jpg
                |-- ...

Test

Download the PartHOE weight and put it in the checkpoints folder.

python parthoe_test.py --cfg config/parthoe.yaml

Train

If you want to retrain the model, you must download the pre-trained VIT-S weight.

python parthoe_train.py --cfg config/parthoe.yaml

Acknowledgement

This work is built upon the open-source project MEBOW and ViTPose. We extend our gratitude to the creators for their outstanding contributions!