Skip to content
/ TTP Public

Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

License

Notifications You must be signed in to change notification settings

harry11162/TTP

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation

This is an official implementation of the NeurIPS 2021 paper: Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation. More details can be found at our project website.

teaser

Preparation

  1. Install dependencies
pip install -r requirements.txt
  1. Make libs

    cd ${PROJECT_ROOT}/lib
    make
  2. Place Penn Action data in data directory. (Instructions on Human3.6M and BBC Pose are coming soon.)

    Your directory tree should look like this:

    ${PROJECT_ROOT}
    └── data
        └── Penn_Action
            ├── frames
            ├── labels
            ├── tools
            └── README
    
  3. Download pretrained model of ResNet-18 and ResNet-50 and place them in models/pytorch/imagenet.

    Your directory tree should look like this:

    ${PROJECT_ROOT}
    └── models
        └── pytorch
            └── imagenet
                ├── resnet18-5c106cde.pth
                └── resnet50-19c8e357.pth
    

Training and Test-time Personalization

Training

python tools/train_joint.py \
   --cfg experiments/penn/joint_res50_128x128_1e-3_comb_attn_tf1_4head.yaml

Run Test-Time Personalization (online)

python tools/test_time_training.py \
   --cfg experiments/penn/ttp_res50_128x128_lr1e-4_online_downsample1_comb_attn_tf1_4head.yaml \
   TEST.MODEL_FILE ${MODEL_FILE}

Run Test-Time Personalization (offline)

python tools/test_time_training.py \
   --cfg experiments/penn/ttp_res50_128x128_lr1e-4_offline_downsample1_comb_attn_tf1_4head.yaml \
   TEST.MODEL_FILE ${MODEL_FILE}

Baseline Model

To train the baseline model for comparison

python tools/train.py --cfg experiments/penn/res50_128x128.yaml

Result

Configs, results and model checkpoints on Human3.6M and BBC Pose are coming soon.

Method TTP Scenario Penn Action Checkpoint
Baseline - 85.233 Google Drive
Ours before TTP 86.283 Google Drive
Ours online 87.660 -
Ours offline 88.633 -

Acknowlegement

TTP is developed based on HRNet. We also incorperate some code from IMM.

About

Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages