Skip to content
Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision
Branch: master
Clone or download
Latest commit 73ead03 May 23, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
flame_model add flame sample May 9, 2019
gif add gif May 20, 2019
input_images add demo images May 14, 2019
smpl_webuser initial commit May 8, 2019
util update license May 15, 2019
LICENSE Initial commit May 7, 2019
README.md Update README.md May 23, 2019
config_test.py update license May 15, 2019
demo.py update license May 15, 2019
requirements.txt neutralize mesh May 9, 2019
run_RingNet.py update license May 15, 2019

README.md

RingNet

alt text

This is an official repository of the paper Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision. The project was formerly referred by RingNet. The codebase consists of the inference code, i.e. give an face image using this code one can generate a 3D mesh of a complete head with the face region. For further details on the method please refer to the following publication,

Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision
Soubhik Sanyal, Timo Bolkart, Haiwen Feng, Michael J. Black
CVPR 2019

More details on our NoW benchmark dataset, 3D face reconstruction challenge can be found in our project page. A pdf preprint is also available on the project page.

Installation

The code uses Python 2.7 and it is tested on Tensorflow gpu version 1.12.0, with CUDA-9.0 and cuDNN-7.3.

Setup RingNet Virtual Environment

virtualenv --no-site-packages <your_home_dir>/.virtualenvs/RingNet
source <your_home_dir>/.virtualenvs/RingNet/bin/activate

Clone the project and install requirements

git clone https://github.com/soubhiksanyal/RingNet.git
cd RingNet
pip install -r requirements.txt
pip install opendr==0.77
mkdir model

Install mesh processing libraries from MPI-IS/mesh.

Download models

  • Download pretrained RingNet weights from the project website, downloads page. Copy this inside the model folder
  • Download FLAME model from here. Copy it inside the flame_model folder. This step is optional and only required if you want to use the output Flame parameters to play with the 3D mesh,i.e., to neutralize the pose and expression and only using the shape as a template for other methods like VOCA (Voice Operated Character Animation).

Demo

RingNet requires a loose crop of the face in the image. We provide two sample images in the input_images folder which are taken from CelebA Dataset.

Output predicted mesh rendering

Run the following command from the terminal to check the predictions of RingNet

python -m demo --img_path ./input_images/000001.jpg --out_folder ./RingNet_output

Provide the image path and it will output the predictions in ./RingNet_output/images/.

Output predicted mesh

If you want the output mesh then run the following command

python -m demo --img_path ./input_images/000001.jpg --out_folder ./RingNet_output --save_obj_file=True

It will save a *.obj file of the predicted mesh in ./RingNet_output/mesh/.

Output FLAME and camera parameters

If you want the predicted FLAME and camera parameters then run the following command

python -m demo --img_path ./input_images/000001.jpg --out_folder ./RingNet_output --save_obj_file=True --save_flame_parameters=True

It will save a *.npy file of the predicted flame and camera parameters and in ./RingNet_output/params/.

Generate VOCA templates

If you want to play with the 3D mesh, i.e. neutralize pose and expression of the 3D mesh to use it as a template in VOCA (Voice Operated Character Animation), run the following command

python -m demo --img_path ./input_images/000013.jpg --out_folder ./RingNet_output --save_obj_file=True --save_flame_parameters=True --neutralize_expression=True

License

Free for non-commercial and scientific research purposes. By using this code, you acknowledge that you have read the license terms (https://ringnet.is.tue.mpg.de/license), understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not use the code. For commercial use please check the website (https://ringnet.is.tue.mpg.de/license).

Referencing RingNet

Please cite the following paper if you use the code directly or indirectly in your research/projects.

@inproceedings{RingNet:CVPR:2019,
title = {Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision},
author = {Sanyal, Soubhik and Bolkart, Timo and Feng, Haiwen and Black, Michael},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
month = jun,
year = {2019},
month_numeric = {6}
}

Contact

If you have any questions you can contact us at soubhik.sanyal@tuebingen.mpg.de and timo.bolkart@tuebingen.mpg.de.

Acknowledgement

  • We thank Raffi Enficiaud and Ahmed Osman for pushing the release of psbody.mesh.
  • We thank Benjamin Pellkofer and Jonathan Williams for helping with our RingNet project website.
You can’t perform that action at this time.