Skip to content
/ DRT Public

SIGGRAPH Asia 2020: Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects

Notifications You must be signed in to change notification settings

lvjiahui/DRT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

30 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DRT

Implementation of Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects.

overview

We introduce a reconstruction method for solid transparent objects. Using a static background with a coded pattern, we establish a mapping between the camera view rays and locations on the background (Environment Matting). Differentiable tracing of refractive ray paths is then used to directly optimize a 3D mesh approximation(visual hull) of the object, while simultaneously ensuring silhouette consistency and smoothness.

Dependencies

Requirements: (tested on ubuntu 16.04)

  • NVIDIA-OptiX-SDK-6.5.0 : Used to find the triangles intersected by each ray path.
  • Meshlab: We use meshlabserver to remesh and calculate average per-vertex distance (Hausdorff Distance). Note that meshlab older than 2020.04 may not support explicit remesh operation.(Tested on MeshLabServer2020.04-linux.AppImage)
  • Python with pytorch, numpy, trimesh, opencv-python, h5py, tqdm and imageio.
  • Ninja build system (needed by pytorch's cpp_extension). Run apt install ninja-build if you use ubuntu.

Captured data

Our example data is captured by camera point grey or cellphone Redmi and released here

Scanned mesh(GT) and visual hull already contained in ./data/

Usage

  • Download Optix-SDK, meshlabserver and our captured data.
  • Set corresponding paths in config.py. Note that you should use absolute path to Optix.
  • Run python optim.py
  • Reconstructed mesh will be saved in ./result/ by default.

Citation

Please cite the paper in your publications if it helps your research:

@article{DRT,
title = {Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects},
author = {Jiahui Lyu and Bojian Wu and Dani Lischinski and Daniel Cohen-Or and Hui Huang},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH ASIA 2020)},
volume = {39},
number = {6},
pages = {195:1--195:13},
year = {2020},
}

About

SIGGRAPH Asia 2020: Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects

Topics

Resources

Stars

Watchers

Forks