A Stereo Approach that Handles the Matting Problem via Image Warping
This project is a end-course project for the Advanced Methods For Computer Vision course at ENS Cachan, under the supervision of Nikos Paragios.
The goal of the project was to re-implement and test the algorithm outlined in the paper A Stereo Approach that Handles the Matting Problem via Image Warping by Michael Bleyer, Margrit Gelautz, Carsten Rother, and Christoph Rhemann.
For those who need the tl;dr, the project was split in the following steps :
- Acquire stereo images
- Rectify them
- Compute the initial disparity map
- Compute an initial segmentation
For more details, see the
Report.tm file (to be opened with TeXmacs)
What you need in order to run this project :
- A working Python 2.7 distribution
- The numpy, scipy, matplotlib and scikits-learn Python packages
- The opencv package and its Python bindings (only for
Just cd into the
src directory and type
make to build the required Python modules.
Then you can run the two test programs,
Rectifying Stereo Images
First, as in a lot of methods in stereo vision, we need rectified images. That is, we need the epipolar lines to coincide with the horizontal scanlines in both the left and the right images.
rectify(left, right) method in
src/rectify.py automatically rectifies a pair of stereo images in argument. It can be called in the terminal, like :
src/$ python rectify.py left.png right.png src/$ ls left.png left-rectified.png right.png right-rectified.png
Disparity map computation
The WarpMat paper mentions the use of another algorithm for the disparity map computation, Simple but effective tree structures for dynamic programming-based stereo matching, by the same authors, Michael Bleyer and Margrit Gelautz, for the initial disparity map computation.
I implemented this algorithm as a Python module,
The algorithm basically runs the same DP algorithm 8 times on the provided images, so the core DP method is written in C, for efficiency, and can be found in the
Another requirement of the WarpMat paper is the segmentation of the input images, which is achieved by using the Mean Shift algorithm from the paper Mean Shift: A robust approach toward feature space analysis, by D. Comanicu and P. Meer.
The corresponding Python module,
meanshift, is a light wrapper around a small part of the source of the EDISON system they implemented, which can be found at this website. These links also were a lot of help in creating this wrapper :
By running the
test.py file, you can see all these stages in action :
warpmat.py file shows how the artificial right view is reconstructed.