The goal of this code is to fit the SMPL model to 3D keypoints. To do that, we do some modifications to the SMPLify method. Our code is based on the original repo of the SMPLify-X project. For instructions on how to get the necessary data please take a look at the README of the original repo which we attach at the end of this file.
The 3D fitting funcionality allows you to fit the SMPL model to 3D keypoints and recover SMPL parameters that are consistent with these keypoints. This can be useful for datasets where only 3D keypoints are available, e.g., Human3.6M or Panoptic. The recovered SMPL parameters can then be used as pseudo ground truth for training, when the true SMPL parameters are not available.
For the 3D fitting, our code expects a pkl file with the same format as the ones we use during training. For instructions on how to generate these files you can refer to the relevant parts in our codebase. To use our code you can run:
python smplifyx/main.py --config=cfg_files/fit_smpl.yaml --model_folder=/path/to/smpl --prior_folder=/path/to/prior --dataset_file=/path/to/pkl
For the fitting, you will need the SMPL model. The argument model_folder indicates the folder where the model files are stored. Please take a look below at the SMPLify-X documentation for more details.
Also, you will need the GMM prior file gmm_08.pkl. The argument prior_folder indicates the folder where this file is stored. You can download this file from the SMPLify webpage. It is located in the folder smplify_public/code/models.
[Project Page] [Paper] [Supp. Mat.]
Software Copyright License for non-commercial scientific research purposes. Please read carefully the terms and conditions and any accompanying documentation before you download and/or use the SMPL-X/SMPLify-X model, data and software, (the "Model & Software"), including 3D meshes, blend weights, blend shapes, textures, software, scripts, and animations. By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.
The original images used for the figures 1 and 2 of the paper can be found in this link. The images in the paper are used under license from gettyimages.com. We have acquired the right to use them in the publication, but redistribution is not allowed. Please follow the instructions on the given link to acquire right of usage. Our results are obtained on the 483 × 724 pixels resolution of the original images.
This repository contains the fitting code used for the experiments in Expressive Body Capture: 3D Hands, Face, and Body from a Single Image.
Run the following command to execute the code:
python smplifyx/main.py --config cfg_files/fit_smplx.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--visualize="True/False"
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDER
--part_segm_fn smplx_parts_segm.pklwhere the DATA_FOLDER should contain two subfolders, images, where the
images are located, and keypoints, where the OpenPose output should be
stored.
To fit SMPL or SMPL+H, replace the yaml configuration file with either fit_smpl.yaml or fit_smplx.yaml, i.e.:
- for SMPL:
python smplifyx/main.py --config cfg_files/fit_smpl.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--visualize="True/False"
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDER- for SMPL+H:
python smplifyx/main.py --config cfg_files/fit_smplh.yaml
--data_folder DATA_FOLDER
--output_folder OUTPUT_FOLDER
--visualize="True/False"
--model_folder MODEL_FOLDER
--vposer_ckpt VPOSER_FOLDERTo visualize the results produced by the method you can run the following script:
python smplifyx/render_results.py --mesh_fns OUTPUT_MESH_FOLDERwhere OUTPUT_MESH_FOLDER is the folder that contains the resulting meshes.
Follow the installation instructions for each of the following before using the fitting code.
- PyTorch Mesh self-intersection for interpenetration penalty
- Download the per-triangle part segmentation: smplx_parts_segm.pkl
- Trimesh for loading triangular meshes
- Pyrender for visualization
The code has been tested with Python 3.6, CUDA 10.0, CuDNN 7.3 and PyTorch 1.0 on Ubuntu 18.04.
If you find this Model & Software useful in your research we would kindly ask you to cite:
@inproceedings{SMPL-X:2019,
title = {Expressive Body Capture: 3D Hands, Face, and Body from a Single Image},
author = {Pavlakos, Georgios and Choutas, Vasileios and Ghorbani, Nima and Bolkart, Timo and Osman, Ahmed A. A. and Tzionas, Dimitrios and Black, Michael J.},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
year = {2019}
}
The LBFGS optimizer with Strong Wolfe Line search is taken from this Pytorch pull request. Special thanks to Du Phan for implementing this. We will update the repository once the pull request is merged.
The code of this repository was implemented by Vassilis Choutas and Georgios Pavlakos.
For questions, please contact smplx@tue.mpg.de.
For commercial licensing (and all related questions for business applications), please contact ps-licensing@tue.mpg.de.