Depth from a polarisation + RGB stereo pair

This repostiory is the implementation of the paper. It contains two part: 1. Using graphical model to correct normal from polarisation images. 2. Estimate final depth with linear equation described by section 6 in the paper. These two parts constitute the pipeline of the paper, but they can run independently. e.g. If you have a coarse depthmap from other source (i.e kinect, multi-view), you can still run the code through the pipe line. Or you have a corrected normal map not from our "Normal correction" you can jump to "Depth estimation" but be specify with other parameters. Please refer the 'data' folder.

If you have any concerned, please fell free to contact me.

Normal correction

This part is implemented under python2.7 with OpenGM library, it takes a corse depth map (In theory, it can take any source of depthmap as long as it aligned with polarisation images. In our paper, the coarse depth is from stereo reconstruction) and polarisation images as input, The output are the corrected normal and estimated specular mask from polarisation information.

Depth estimation

This part is implemented under Matlab. It takes corrected normal, (But can be any kind of "guide" surface normal), estimated specular mask, polarisation images, light source and camera matrix. It estimate the albedo and depth of the object.

Tools

In order to install OpenGM please refer Installing OpenGM with Python wrapper, or by following command(Only support python2.7 and test on linux only)

  sudo apt-get install libopengm-bin libopengm-dev libopengm-doc python-opengm python-opengm-doc

Estimate polarisation information from polarimetric images

Citation

If you use this code in your research, please cite the following paper:

D. Zhu and W.A.P. Smith. Depth from a polarisation + RGB stereo pair. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Bibtex:

@inproceedings{zhu2019depth,
  title={Depth from a polarisation + RGB stereo pair},
  author={Zhu, Dizhong and Smith, William AP},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019}
}