Skip to content
CS283 Project - Pipeline for Improving Accuracy of Hand Tracking
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Pipeline for Improving Accuracy of Hand Tracking

The goal of this project is to create a simple hand gesture detection system, using relatively few images to train our hand classifier. Given a poorly trained Haar Cascade Classifier (250 positive samples and 100 negative samples) to recognize hands, this project assembles a pipeline to improve the quality of the tracking. These steps include:

  1. Face detection and removal of faces.
  2. Background subtraction.
  3. Use a simplified Kalman-Filter-esque technique to estimate the bounding box of the hand. This assumes that a hand moves in a smooth manner.
  4. Use our hand classifier to detect the largest hand within the bounding box.
  5. Compute the optical flow of points within the bounding box using Lucas-Kanade.
  6. Use the optical flow and the measured position of the hand to correct our Kalman-Filter estimate.


  • OpenCV 2.4.3
  • Tornado (Python) 2.4.1
  • Python 2.7
  • Websockets
  • Chrome

How to Run the Code

Running the Pipeline

You must have OpenCV 2.4.1+ installed and Python 2.7. To run the detection pipeline without the maps application, run:


To choose a different pipeline, choose one of the following, e.g.:

python --pipeline=nofacekalman

To see all the pipeline types and more flags, use the -h flag:

python -h

Running the Maps Application

To run the server, you must install Tornado (Python) 2.4.1. To start the server, specify the port (default is 8888):

python --port=8888 accepts the same command line arguments as To open the client on the server, edit static/settings.js to match the port specified on the server. static/settings.js should look like this:

MotionApp = {
    HOST: "localhost",
    PORT: 8888,
    WIDTH: 320,
    HEIGHT: 240,
    SCALE_FACTOR: 0.5,

Then visit localhost:8888 in Chrome (you must be using the latest version of Chrome and have Websockets enabled, see chrome://flags). To scroll around, face the palm of your hand, fingers together, towards the camera, and move your hand around!

You can’t perform that action at this time.