MVA -- Advanced Computer Vision Course -- Project on Stereo Matting
C++ C Python
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
images
presentation
src
.gitignore
Readme.md
Report.tm

Readme.md

A Stereo Approach that Handles the Matting Problem via Image Warping

This project is a end-course project for the Advanced Methods For Computer Vision course at ENS Cachan, under the supervision of Nikos Paragios.

The goal of the project was to re-implement and test the algorithm outlined in the paper A Stereo Approach that Handles the Matting Problem via Image Warping by Michael Bleyer, Margrit Gelautz, Carsten Rother, and Christoph Rhemann.

For those who need the tl;dr, the project was split in the following steps :

  1. Acquire stereo images
  2. Rectify them
  3. Compute the initial disparity map
  4. Compute an initial segmentation
  5. WarpMat

For more details, see the Report.tm file (to be opened with TeXmacs)

What you need in order to run this project :

Just cd into the src directory and type make to build the required Python modules. Then you can run the two test programs, test.py and warpmat.py.

Rectifying Stereo Images

First, as in a lot of methods in stereo vision, we need rectified images. That is, we need the epipolar lines to coincide with the horizontal scanlines in both the left and the right images.

Scanlines in a rectified image pair

The rectify(left, right) method in src/rectify.py automatically rectifies a pair of stereo images in argument. It can be called in the terminal, like :

src/$ python rectify.py left.png right.png
src/$ ls
   left.png left-rectified.png right.png right-rectified.png

Disparity map computation

The WarpMat paper mentions the use of another algorithm for the disparity map computation, Simple but effective tree structures for dynamic programming-based stereo matching, by the same authors, Michael Bleyer and Margrit Gelautz, for the initial disparity map computation.

I implemented this algorithm as a Python module, simpletree.

The algorithm basically runs the same DP algorithm 8 times on the provided images, so the core DP method is written in C, for efficiency, and can be found in the simpletree.dp module.

Disparity map for the Tsukuba image pair

Image segmentation

Another requirement of the WarpMat paper is the segmentation of the input images, which is achieved by using the Mean Shift algorithm from the paper Mean Shift: A robust approach toward feature space analysis, by D. Comanicu and P. Meer.

The corresponding Python module, meanshift, is a light wrapper around a small part of the source of the EDISON system they implemented, which can be found at this website. These links also were a lot of help in creating this wrapper :

Tests

By running the test.py file, you can see all these stages in action :

Disparity map for the Tsukuba image pair

Also, the warpmat.py file shows how the artificial right view is reconstructed.

Disparity map for the Tsukuba image pair