Code for the upcoming CVPR 2016 paper
Jupyter Notebook Python
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Structured Prediction of Unobserved Voxels from a Single Depth Image

  author = {Michael Firman and Oisin Mac Aodha and Simon Julier and Gabriel J Brostow},
  title = {{Structured Completion of Unobserved Voxels from a Single Depth Image}},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2016}

Downloading the dataset

The tabletop dataset can be downloaded from:

This is a 395MB zip file. You will have to change some of the paths in src/pipeline/ to the location you have extracted the dataset to.

The voxel regions used to evaluate each of the scenes can be downloaded from here:

Getting started with the tabletop dataset

An example iPython notebook file loading a ground truth TSDF grid and plotting on the same axes as a depth image is given in src/examples/Voxel_data_io_example.ipynb

Code overview

The code is roughly divided into three areas:

  1. src/common/ is a Python module containing all the classes and functions used for manipulation of the data and running of the routines. Files included are:

    • - classes for RGBD images and videos
    • - a camera class, enabling points in 3D to be projected into a virtual depth image and vice versa
    • - a class for 3D mesh data, including IO and marching cubes conversion
    • - classes for 3D voxel grids, including various manipulation routines and ability to copy data between grids at different locations in world space
    • - classes for reconstructing 3D volumes from extrinsicly calibrated depth images.
    • - classes for computing normals and other features from depth images
    • - structured RF class
    • - class which contains a voxel grid and one or more images with corresponding coordinate frames.
    • - class for storing and predicting voxlets, and for doing the final reconstruction of the output voxel grid.
  2. src/pipeline/ - Contains scripts for loading data, performing processing and saving files out.

  3. src/examples/ - iPython notebooks containing examples of use of the data and code.


I have run this code using Python 2 on a fairly up-to-date version of Anaconda on Ubuntu 14.04. If you have a newer version of scipy, you will need to install weave seprately, which has been moved to its own project.

How to run the pipeline on the tabletop dataset

Navigate to src/pipeline

>> python
>> python
>> python
>> python
>> python

How to run the pipeline on the synthetic NYU dataset

Navigate to src/pipeline

>> python training_params_nyu_silberman.yaml
>> python training_params_nyu_silberman.yaml
>> python training_nyu_silberman.yaml
>> python testing_params_nyu_silberman.yaml
>> python testing_params_nyu_silberman.yaml