Skip to content
We propose to exploit nearby and farther clues available from image and disparity domains to obtain a more accurate confidence estimation. While local information is very effective for detecting high frequency patterns, it lacks insights from farther regions in the scene. On the other hand, enlarging the receptive field allows to include clues f…
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
images
model
README.md

README.md

LGCNet-Tensorflow

Tensorflow implementation of a local-global framework for confidence estimation.

Beyond local reasoning for stereo confidence estimation with deep learning

Fabio Tosi, Matteo Poggi, Antonio Benincasa and Stefano Mattoccia
ECCV 2018

Qualitative results on KITTI

Alt text

Example of confidence estimation. (a) Reference image from KITTI 2015 dataset, (b) disparity map obtained with MC-CNN, (c) confidence estimated with a local approach (CCNN) and (d) the proposed local-global framework, highlighting regions on which the latter method provides more reliable predictions (red bounding boxes).

For more details: pdf

Training

For training, the KITTI input images have been padded to 384x1280.

The training file should be a .txt file in which each line contains: [path_image_left];[path_image_disparity];[path_image_groundtruth]. Disparity maps and groundtruth as 16 bit images.

You can train the local network as follows:

python ./model/main.py --is_training True --epoch 14 --batch_size 128 --patch_size 9 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model [CCNN, EFN, LFN] 

Use the --model argument to choose the architecture:

Similarly, you can train the global network (ConfNet):

python ./model/main.py --is_training True --epoch 1600 --batch_size 1 --crop_height 256 --crop_width 512 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model ConfNet 

Finally, you can load weights from the local and global networks and train LGCNet thereafter:

python ./model/main.py --is_training True --epoch 14 --batch_size 128 --patch_size 9 --dataset [path_training_file] --initial_learning_rate 0.003 --log_directory [path_log] --model_name model --model LGC --checkpoint_path [path_checkpoint_ConfNet] [path_checkpoint_CCNN/LFN] --late_fusion

Use --late_fusion flag to set LFN as local network. Otherwise, disable it to use CCNN.

Warning: set checkpoint_CCNN/LFN accordingly.

Testing

The testing file should be a .txt file in which each line contains: [path_image_left];[path_image_disparity]. Disparity maps as 16 bit images.

If you want to test the local network or the global network indipendently:

python ./model/main.py --is_training False --batch_size 1 --dataset [path_testing_file] --checkpoint_path [path_checkpoint] --output_path [path_output] --model [CCNN, EFN, LFN, ConfNet]

For testing LGCNet, instead, you can run:

python ./model/main.py --is_training False --batch_size 1 --dataset [path_testing_file] --checkpoint_path [path_checkpoint_ConfNet] [path_checkpoint_CCNN/LFN] [path_checkpoint_LGC] --output_path [path_output] --model LGC --late_fusion

Pretrained models

You can download the pre-trained models, trained on 20 images of KITTI 12 dataset for AD-CENSUS, SMG and MC-CNN stereo algorithms, here:

Google Drive

Requirements

This code was tested with Tensorflow 1.4, CUDA 8.0 and Ubuntu 16.04.

You can’t perform that action at this time.