Geometry and Context Network (On-Going Project)

A Keras implementation of GC Network by HungShi Lin(hl2997@columbia.edu). The paper can be found here. I do some modifications by adding a linear output function and enable training highway block at the second stage.

Issue

1. Model performance can't acheive that in the original paper.

Update (10/06/2017)

Model can be trained with image with size (256, 512)

Software Requirement

tensorflow(install from here), keras(install from here)

Data used for training model

I trained my model with drivingfinalpass dataset, which contains more than 4000 stereo images with 2 epochs.

Preprocessing

We crop training patches with size of 256x256 (different from that in the paper) from training images and normalize each channel.

Download

Run the following command:

  git clone https://github.com/LinHungShi/GCNetwork.git

Two ways to download driving dataset:

1. create subdirectories sceneflow/driving in data, download and tar driving_final pass and driving_disparity from here.
2. You can also issue command

  “sh download.sh” 

which will create subdirectories and download datasets.

Train the model by running:

Run the following command:

  python train.py

Predict the data with test.py

1. create a directory, which contains two subdirectories -- left and right.
2. Run the following command

  python test.py -data <path/to/directory> -wpath <path/to/weight> [option]

3. The default file will be saved as npy file named prediction.npy, you can replace it with -pspath when issuing the above command.

(Optional) Specify the pretrained weight by

1. Set it in train_params.py
2. python train.py -wpath

Something you might want to do

1. To enable training with Monkaa dataset, a. Download Monkaa dataset from previous link.

   b. Create a directory in data, which has the name as monkaa_root in src/environment.json.

   c. Create a subdirectory, which has the name as monkaa_train in src/environment.json.

   d. Create a subdirectory, which has the name as monkaa_label in src/environment.json.

2. All hyperparameters used for building the model can be found in src/hyperparams.json

Reference :

Kendall, Alex, et al. "End-to-End Learning of Geometry and Context for Deep Stereo Regression." arXiv preprint arXiv:1703.04309 (2017).