Skip to content
Welcome to the code repository of CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation. CVPR 2018
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cpp using global definition THREADS_NUM_USED for control OMP threads Mar 19, 2019
mb toy training dataset for code testing Mar 19, 2019
pylibs CBMV Jun 15, 2018
.gitignore ignore cpp/build Mar 19, 2019
.gitignore_global CBMV Jun 16, 2018
LICENSE Initial commit Mar 20, 2018
README.md specify sklearn version Mar 19, 2019
disp.pfm CBMV Jun 15, 2018
main.py add args.n_jobs to control the number of jobs to run in parallel for … Mar 19, 2019
progressbar.py CBMV Jun 15, 2018
run_test_cbmv.sh
run_train_cbmv.sh add args.n_jobs to control the number of jobs to run in parallel for … Mar 19, 2019
test.py minor update for local expansion Mar 19, 2019
train.py

README.md

CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation

CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation code repository. If you use this code please cite our paper CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation.

@inproceedings{batsos2018cbmv,
  title={CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation},
  author={Batsos, Konstantinos and Cai, Changjiang and Mordohai, Philipos},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2018}
}

The repository contains the code, models and procudures for training and testing. The code includes the cost-optimization and post-processing of MC-CNN Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches as implemented by Jure Zbontar, and modified to interface with python. We aslo incorporate, as an alternative cost-optimization method, the code from Continuous 3D Label Stereo Matching using Local Expansion Moves.

Links

Training Data

CBMV model: the trained random forest model, with version 0.17.1 sklearn.

Run CBMV

To run CBMV you need the following python packages:

Python

  • numpy
  • scipy
  • sklearn (version 0.17.1)
  • cPickle

Much of CBMV code is written as C++ libraries and interface with python via boost python. To compile the C++ libraries you will need the following:

C++

  • OpenCV 3.0
  • Boost ( boost python )
  • CUDA

Compilation

Assuming this repository is rooted at "~/cbmv-cvpr18/", the OpenCV library is installed at /usr/local/opencv-3.2.0.

    1. To compile matchers, featextract and rectification:
      We provide both makefile and CMakeLists.txt(preferred) for compilation. If you have installed OpenCV at different directory, please update the line g++ -I/usr/local/opencv-3.2.0/include/ ... in the file */Debug/subdir.mk for including header files; and modify the line g++ ... -L/usr/local/opencv-3.2.0/lib ... in the file */Debug/makefile for library linking. Or to modify the line set(OpenCV_DIR "/usr/local/opencv-3.2.0/share/OpenCV/") in the file CMakeList.txt.
# 1) Method 1 : using CMake, will generate the libraries in the directory `~/cbmv-cvpr18/cpp/lib`.
cd ~/cbmv-cvpr18/cpp/ && mkdir build && cd build && cmake .. && build 

# 2) Method 2 : using makefile
#To compile `matchers` for four basic matchers:
cd ~/cbmv-cvpr18/cpp/matchers/Debug && make clean && make
# To compile `featextract` for feature extraction:
cd ~/cbmv-cvpr18/cpp/featextract/Debug && make clean && make
# To compile `rectification` for rectifying stereo image pair:
cd ~/cbmv-cvpr18/cpp/rectification/Debug && make clean && make
    1. To compile post for postprocessing:
      This part contains the GPU code. We assume the CUDA was installed at /usr/local/cuda-8.0/. If not, please modify the line CUDA_LIBS = /usr/local/cuda-8.0/lib64 in the file */post/makefile.
cd ~/cbmv-cvpr18/cpp/post/ && make clean && make
    1. To compile localexp for Local Expansion Moves:
      This part requires the Maxflow code by Boykov and Kolmogorov [Code v3.01] [Code v3.04]. For your convenience, we already include and confiure it. But please note they are freely available for research purposes only. You could also check the Local Expansion Move repository for the license. For localexp compilation, we provide the CMakeLists.txt file. Run the following
# will generate the libraries in the directory `*/localexp/lib`.
cd ~/cbmv-cvpr18/cpp/localexp && mkdir build && cd build && cmake .. && make

You can change the OMP threads number defined by #define THREADS_NUM_USED 16 at file ~/cbmv-cvpr18/cpp/paramSetting.hpp, for efficient parallel computation.


After compiling the C++ libraries, downloading the required python packages and precomputed model (Note: pay attention to the version of sklearn), CBMV can be executed using the following command

python main.py  --l ./datasets/ArtL/im0.png --r ./datasets/ArtL/im1.png --w 694 --h 554 --d 128 --model ./modelall_4m.rf --prob_save_path ./

For a complete list of arguments please see tha main.py file. We provide a script run_test_cbmv.sh to run CBMV for testing and see more details in section Disparity Estimation.

Training

The above link includes the data we used to trained CBMV. To train CBMV you can use the following command:

python main.py --train --data_path "./datasets/" --train_add ./mb/additional.txt --train_set ./mb/trainall.txt --model ./models/model.rf

The *.txt files specify the training sets used during the training process. For a complete list of arguments please see tha main.py file. We also provide a script run_train_cbmv.sh to train CBMV.

Disparity Estimation

Giveing the cost volume, there are two methods for cost-optimization in order to generate the disparity map as output.

  • Post-processing used by MC-CNN: see the code ~/cbmv-cvpr18/cpp/post/post.cu for details.
  • Local Expansion Moves: see the codes ~/cbmv-cvpr18/cpp/localexp/local_exp_stereo.cpp for more details. You can also modify the hyperparameters defined in function __postprocessing_localExp(...) in the file ~/cbmv-cvpr18/test.py.
You can’t perform that action at this time.