Improving 2D Human Pose Estimation across Unseen Camera Views with Synthetic Data

The official repository of the RePoGen paper.

Project webpage | ArXiv

Table of Contents

• Description
• News
• Installation
• Datasets
• Model
• Licence
• Acknowledgements
• Citation & Contact

Description

RePoGen (RarE POses GENerator) is a method for synthetic data generation using the SMPL-X. RePoGen generates humans in very rare poses using the estimation of rotation distribution for each SMPL-X joint. The generated poses are used to augment the COCO dataset to improve performance on extreme views.

News

• 17 July 2023: Code released.
• 17 July 2023: Datasets and weights released.
• 16 July 2023: The Readme along with description is available. You can read the paper and see its website. The code is comming soon.

Installation

The RePoGen is installed from the source:

git clone https://github.com/MiraPurkrabek/RePoGen
python setup.py install

The code requires psbody.mesh library from MPI-IS (link). For easiest installation, please use our fork of the repository and install it from source. For more details, see issue #12.

You also have to download the SMPL-X model. See the instructions here.

Once you install everything right, data are generated by the script here.

Datasets

Introduced datasets are available to download on the project webpage. The RePo dataset was manually annotated while the RePoGen dataset was generated by the RePoGen method. We also give you the code in this repository to generate your own synthetic dataset.

Model

If you are only interested in the pre-trained weights, we release the best model trained on the COCO+RePoGen dataset here. The weights are for the ViTPose-s with classic decoder.

Licence

Please read carefully the terms and conditions and any accompanying documentation before you download and/or use the RePoGen model, data and software, (the "Model & Software"). By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.

Please also note that the RePoGen depends on the SMPL-X which is free to use only for non-commercial scientific research purposes. For more, see their homepage.

Acknowledgements

The code builds on the SMPL-X repository.

For experiments, we used the COCO dataset and the PoseFES dataset.

Citation & Contact

The code was implemented by Miroslav Purkrábek.

For questions, please use the Issues of Discussion.

@misc{purkrábek2023improving,
  title={Improving 2D Human Pose Estimation across Unseen Camera Views with Synthetic Data}, 
  author={Miroslav Purkrábek and Jiří Matas},
  year={2023},
  eprint={2307.06737},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}