Volumetric TSDF Fusion of Multiple Depth Maps

CUDA/C++ code to fuse multiple registered depth maps into a projective truncated signed distance function (TSDF) voxel volume, which can then be used to create high quality 3D surface meshes and point clouds. Tested on Ubuntu 14.04 and 16.04.

Looking for an older version? See here.

This repository is a part of Andy's Code Collection.

Change Log

• Nov. 1, 2017. Bug fix: tsdf2mesh.m now properly generates a mesh in camera coordinates instead of voxel coordinates.
• Oct. 30, 2017. Notice: changed default weight threshold for SaveVoxelGrid2SurfacePointCloud in demo code to enable creating point cloud visualizations with only one depth frame.
• Aug. 30, 2017. Bug fix: remove deprecated offsets from surface distance compute during integration.

Requirements

• NVIDA GPU with CUDA support
• OpenCV (tested with OpenCV 2.4.11)

Demo

This demo fuses 50 registered depth maps from directory data/rgbd-frames into a projective TSDF voxel volume, and creates a 3D surface point cloud tsdf.ply, which can be visualized with a 3D viewer like Meshlab.

Note: Input depth maps should be saved in format: 16-bit PNG, depth in millimeters.

./compile.sh # compiles demo executable
./demo # 3D point cloud saved to tsdf.ply and voxel grid saved to tsdf.bin

[Optional] This demo also saves the computed voxel volume into a binary file tsdf.bin. Run the following script in Matlab to create a 3D surface mesh mesh.ply, which can be visualized with Meshlab.

tsdf2mesh; % 3D mesh saved to mesh.ply

Usage in Spartan

The executable demo accepts two command line arguments. The first is the full path to the the folder where the depth images are stored, the second is the path to the camera-intrinsics.txt file. The images folders structure should be

images
 -- 000001_depth.png
 -- 000001_pose.txt
 .
 .

The executable outputs two files, tsdf.bin and tsdf_pointcloud.ply, both in the images folder that was passed in. tsdf.bin is a binary file containing the actual TSDF, can be converted to a mesh using code in spartan. tsdf_pointcloud.ply is a pointcloud sampled from the TSDF implicit surface and encoded as a ply file.

Seen in

• 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions (CVPR 2017)
• Semantic Scene Completion from a Single Depth Image (CVPR 2017)
• Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images (CVPR 2016)

References

• A Volumetric Method for Building Complex Models from Range Images (SIGGRAPH 1996)
• KinectFusion: Real-Time Dense Surface Mapping and Tracking (ISMAR 2011)
• Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images (CVPR 2013)