Training and Prediction with the NYU Depth v2 Dataset

Step 1: Build CURFIL

Build and install CURFIL as described in the README.

Step 2: Download

Download the labeled dataset nyu_depth_v2_labeled.mat (2.8 GB) from http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html, and the train/test split file splits.mat (2.6 kB) from http://cs.nyu.edu/~silberman/projects/indoor_scene_seg_sup.html.

Step 3: Convert

Convert the dataset into a format that CURFIL can read. We provide a python script (scripts/NYU/convert.py) to convert the Matlab file of the NYU Depth v2 dataset to the CURFIL dataset format, which is a set of PNG images.

The python scripts requires the modules h5py, scipy, scikit-image, numpy, pypng and joblib.

We recommend to use easy_install or pip in a virtualenv to install the python modules. Example:

easy_install --prefix=~/.local h5py scipy scikit-image numpy pypng joblib

Hint: Manually create the non-existing directories if the easy_install command fails.

Now you can run the actual conversion with:

python -O scripts/NYU/convert.py nyu_depth_v2_labeled.mat splits.mat <NYU_DIRECTORY>

It uses joblib to spawn multiple processes that convert the dataset in parallel. Still, it can take some time on a slow machine. <NYU_DIRECTORY> will contain the converted training and testing images in two subdirectories training and testing, respectively.

Step 4: Training

We continue to train a random forest after the dataset is converted. Training is done using the curfil_train binary. It requires to specify a set of parameters that are described in Training Parameters.

The following example shows training with parameters that we found to yield good segmentation accuracy on the NYU Depth v2 dataset.

curfil_train --folderTraining <NYU_DIRECTORY>/training \
             --trees 3 \
             --samplesPerImage 4537 \
             --featureCount 5729 \
             --minSampleCount 204 \
             --maxDepth 18 \
             --boxRadius 111 \
             --regionSize 3 \
             --numThresholds 20 \
             --ignoreColor '0,0,0' \
             --outputFolder <OUTPUT_DIR>

if your GPU has a very large memory, this might give you CUDA errors. In this case, limit the --imageCacheSize.

The statement will train a random forest with three trees on the GPU. The three trees of the random forest is the trained model which is stored in <OUTPUT_DIR> as compressed JSON files, one file per tree.

Step 5: Prediction

curfil_predict <PREDICTION_DIR> <NYU_DIRECTORY>/testing <OUTPUT_DIR>/tree*.gz

The prediction will load the tree files <OUTPUT_DIR>/tree{0-2}.json.gz and perform a dense pixel-wise prediction of all test images in <NYU-DIRECTORY>/testing. It will write the prediction result to <PREDICTION_DIR> and will log the segmentation accuracy measures.