Learning Based Calibrated Photometric Stereo for Non-Lambertian Surface (ECCV 2018)
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
datasets
eval
images
models
options
scripts
utils
.gitignore
LICENSE.txt
README.md
main.py
test_utils.py
train_utils.py

README.md

PS-FCN

PS-FCN: A Flexible Learning Framework for Photometric Stereo, ECCV 2018,
Guanying Chen, Kai Han, Kwan-Yee K. Wong

This paper addresses the problem of learning based photometric stereo for non-Lambertian surface:

Dependencies

PS-FCN is implemented in PyTorch and tested with Ubuntu 14.04, please install PyTorch first following the official instruction.

  • Python 2.7
  • PyTorch (version >= 0.40)
  • numpy
  • scipy
  • CUDA-8.0

Overview:

We provide:

  • Datasets: Blobby dataset (4.7 GB), Sculpture dataset (19 GB)
  • Trained models (trained on both the Blobby dataset and the Sculpture dataset with a per-sample input number of 32):
    • PS-FCN for calibrated photometric stereo
    • UPS-FCN for uncalibrated photometric stereo
  • Code to test on DiLiGenT main dataset
  • Code to train a new model

Testing

Download the trained models

sh scripts/download_pretrained_models.sh

Test on the DiLiGenT main dataset

# Download DiLiGenT main dataset
sh scripts/prepare_diligent_dataset.sh

# Test PS-FCN on DiLiGenT main dataset using all of the 96 image-light pairs
CUDA_VISIBLE_DEVICES=0 python eval/run_model.py --retrain data/models/PS-FCN_B_S_32.pth.tar --in_img_num 96

# Test UPS-FCN on DiLiGenT main dataset only using images as input 
CUDA_VISIBLE_DEVICES=0 python eval/run_model.py --retrain data/models/UPS-FCN_B_S_32.pth.tar --in_img_num 96 --in_light

Training

To train a new PS-FCN model, you have to follow the following steps:

Download the training data

# The total size of the zipped synthetic datasets is 4.7+19=23.7 GB 
# and it take some times to download and unzip the datasets.
sh scripts/download_synthetic_datasets.sh

Call main.py to train PS-FCN or UPS-FCN

# Train PS-FCN on both synthetic datasets using 32 images-light pairs
CUDA_VISIBLE_DEVICES=0 python main.py --concat_data --in_img_num 32

# Train UPS-FCN on both synthetic datasets using 32 images
CUDA_VISIBLE_DEVICES=0 python main.py --concat_data --in_img_num 32 --in_light --item uncalib

# Please refer to options/base_opt.py and options/train_opt.py for more options

# You can find checkpoints and results in data/Training/

FAQ

Q1: How to test PS-FCN on other dataset?

  • You have to implement a customized Dataset class to load your data. Please refer to datasets/DiLiGenT_data.py for an example that loads the DiLiGenT main dataset. Precomputed results on DiLiGenT main/test dataset, Gourd&Apple dataset, Light Stage Dataset and Synthetic Test dataset are available upon request.

Q2: Which eight sculpture shapes were used in rendering the training datasets? Why?

  • Normal maps of the eight selected sculpture shapes are shown in the below figure (filenames can be found in this link). We chose these 8 shapes for their high quality meshes and complex geometry. Other complex 3D objects can also be used.

Q3: What should I do if I have problem in running your code?

  • Please create an issue if you encounter errors when trying to run the code. Please also feel free to submit a bug report.

Citation

If you find this code or the provided data useful in your research, please consider cite:

@inproceedings{chen2018ps,
  title={PS-FCN: A Flexible Learning Framework for Photometric Stereo},
  author={Chen, Guanying and Han, Kai and Wong, Kwan-Yee K.},
  booktitle={ECCV},
  year={2018}
}