Explore the docs · Request Feature
This project currently has two main features.
The first functionality is to estimate a homography between two input image frames. This is achieved by finding key points in each image, finding key point matches between the images, and running RANSAC to determine inliers and outliers in the matches.
The second functionality is to extend the first feature to operate over an entire sequence of frames, or an input video. This outputs a new video highlighting inliers and outliers across frames as well as homographies relating each pair of consecutive frames.
Homographies
In computer vision, a homography is a transformation that describes the relationship between any two images (or photographs) of the same plane in space. More mathematically (in projective geometry), a homography is an isomorphism of projective spaces, and have been historically used to explain and study the difference in appearance of two planes observed from different points of view. Homographies can be represented as a 3 x 3 matrix which can be left-multiplied with any homogeneous point in the original image to describe where that point lies in the transformed image.
Applications of homographies include removing perspective distortion (computer vision), rendering textures (computer graphics), and computing planar shadows (computer graphics). In this project, homographies are used to compute the camera motion - namely the rotation and translation - between two images.
RANSAC algorithm
The RANdom SAmple Consensus (RANSAC) algorithm is a general parameter estimation approach to compensate for a large proportion of outliers in the data. In this application, the input data to RANSAC is the collection of keypoint matches between consecutive frames, and the algorithm picks out matches which are true matches (inliers) versus false matches (outliers).
Algorithm (from http://www.cse.yorku.ca/~kosta/CompVis_Notes/ransac.pdf):
- Select randomly the minimum number of points required to determine the model parameters.
- Solve for the parameters of the model.
- Determine how many points from the set of all points fit with a predefined tolerance ε.
- If the fraction of the number of inliers over the total number points in the set exceeds a predefined threshold τ, re-estimate the model parameters using all the identified inliers and terminate.
- Otherwise, repeat steps 1 through 4 (maximum of N times).
Since a homography can be computed from just 4 tie points, step 1 requires randomly choosing 4 tie points between the images.
Input images
Identify keypoints
Determine matches
Use RANSAC to find inliers (green) & outliers (red)
| 1.00000000e+00 | -1.84873558e-12 | 2.91736753e-09 |
| 2.39456865e-12 | 1.00000000e+00 | -1.08356205e-08 |
| 1.20185168e-15 | -5.76888806e-16 | 1.00000000e+00 |
Input and output video
To get a local copy up and running follow these simple steps.
- Clone the repo
git clone https://github.com/dastratakos/Homography-Estimation.git- Add images and videos. Images should go in the directory
input/images/[dir]and videos should go in the directoryinput/videos/[dir], where[dir]is a 2 digit unique identifier within either directory.
- Here is an example of running homography estimation on two image frames,
which are stored as
input/images/01/1.pngandinput/images/01/2.png.
python imageAnalysis.py -d 01- This is an example of running homography estimation on a full video, which
is stored as
input/videos/02/IMG_1554.MOV.
python videoAnalysis.py -d 02- Reformat output of video analysis to collect homographies for each pair of consecutive frames in a CSV or JSON file.
- In video analysis, use the estimated homographies to create a "tripod stabilization" effect.
- Fine tune inlier/outlier points.
See the open issues for a more detailed list of proposed features (and known issues).
Distributed under the Apache 2.0 License. See the LICENSE for more information.
Dean Stratakos - dstratak@stanford.edu