Skip to content

NVlabs/two-shot-brdf-shape

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Two-shot Spatially-varying BRDF and Shape Estimation

Project Page | Video | Paper | Data

Tensorflow implementation using Tensorpack for predicting the shape, illumination and BRDF of an object using two handheld images.

Two-shot Spatially-varying BRDF and Shape Estimation
Mark Boss1, Varun Jampani2, Kihwan Kim2, Hendrik P. A. Lensch1, Jan Kautz2
1University of Tübingen, 2NVIDIA

License

Copyright (C) 2020 NVIDIA Corporation. All rights reserved.

This work is made available under the NVIDIA Source Code License.

Setup

This project uses Pipenv for dependency management.

conda env create -f environment.yml

You are now in the correct virtual environment shell.

Inference

As the weights are comparably small, they are included in the weights directory of this git. This git also includes an inference_images folder. You can places images inside the example folder, where each sample is placed in its folder.

Performing an inference on own data is then performed with the infer.py script:

python infer.py --data inference_images -s weights/shape_graph.pb -i weights/illumination_graph.pb -b weights/brdf_graph.pb -j weights/joint_graph.pb --gpu 0

Notice that per default, a Low Dynamic Range sRGB image is expected. If the data is available in linear color space or High Dynamic Range format, the --linear or --hdr option can be used respectively.

Training

Performing full retraining is due to the sequential nature of the pipeline, also a sequential process. Each network first needs to be trained, then a complete inference step on the dataset needs to be executed.

If not already done, download the dataset first and extract it.

python train_stage.py shape --data PATH_TO_DATASET/training --save TRAINDIR/brdf --gpu 0
python advance_stage.py shape --data PATH_TO_DATASET/training -w TRAINDIR/brdf/compact.pb --gpu 0

python train_stage.py illumination --data PATH_TO_DATASET/training --save TRAINDIR/illumination --gpu 0
python advance_stage.py illumination --data PATH_TO_DATASET/training -w TRAINDIR/illumination/compact.pb --gpu 0

python train_stage.py brdf --data PATH_TO_DATASET/training --save TRAINDIR/brdf --gpu 0
python advance_stage.py brdf --data PATH_TO_DATASET/training -w TRAINDIR/brdf/compact.pb --gpu 0

python train_stage.py joint --data PATH_TO_DATASET/training --save TRAINDIR/joint --gpu 0
python advance_stage.py joint --data PATH_TO_DATASET/training -w TRAINDIR/joint/compact.pb --gpu 0

After the training is done a compact.pb file exists in the specified training directory. This file can then be used for the inference.

Configuration

Each stage of the training can be altered using command line parameters. Consult the --help option of the train_stage.py script for a detailed explanation. Additionally, the render config can be altered in the config/render_config.json file.

Citation

@inproceedings{Boss2020-TwoShotShapeAndBrdf,
    author = {Boss, Mark and Jampani, Varun and Kim, Kihwan and Lensch, Hendrik P.A. and Kautz, Jan},
    title = {Two-shot Spatially-varying BRDF and Shape Estimation},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2020}
}

About

Two-shot Spatially-varying BRDF and Shape Estimation

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published