Hand tracking on a tabletop display using Kinect
Java Ruby
Switch branches/tags
Nothing to show
Latest commit ffdeeef Sep 25, 2014 @ushadow Update README.md
Failed to load latest commit information.
config Fixed a bug of NullPointerException. Jun 20, 2014
data/calibration Fixed a bug of NullPointerException. Jun 20, 2014
lib Changed to use config properties instead of command line options. Aug 11, 2011
src/edu/mit/yingyin Fixed a bug of NullPointerException. Jun 20, 2014
test/edu/mit/yingyin Added plotting of clusters. Mar 22, 2013
test_data WIP: cleaning up calibration code. Apr 10, 2012
.classpath Updated README.md Dec 3, 2012
.gitignore Switch back to use findResource to create scheduler. Apr 6, 2013
.project Removed scala nature. Nov 5, 2012
README.md Update README.md Sep 25, 2014
Rakefile Added clustering code. Jan 31, 2013
build.xml Merge branch 'master' of git.pwnb.us:ushadow/tabletop_kinect Apr 25, 2013
makefile Refactored code. Nov 26, 2012


This code is part of the research project for gestural input on a tabletop display. One Microsoft's Kinect is used for tracking hands. The Kinect is placed above the tabletop about 1.2m above. This [picture] (http://people.csail.mit.edu/yingyin/resources/image/tabletop.png) shows the setup. Currently, the code is only tested with this setup. Certain parameters are set based on the assumptions according to this setup, e.g., the distance of the tabletop from the Kinect. A different setup may cause some issues.


  • Currently developing on Ubuntu 12.04 with Sun Java 7. Tested on Windows 7.


Install OpenNI framework and the corresponding Kinect driver

  • [OpenNI] (https://github.com/OpenNI/OpenNI) (Version unstable version - May 7th 2012)

  • [SensorKinect] (https://github.com/avin2/SensorKinect) PrimeSense Sensor Module for OpenNI (VVersion unstable version - May 15th 2012) Kinect Mod:

    • Currently tested with unstable branch commit 2d139677f26c06fcd9fc8d4a7db033d70021624b
    • Follow the Linux installation instructions in README
    • Or use the binaries available at http://www.openni.org/Downloads/OpenNIModules.aspx. Choose "OpenNI Compliant Hardward Binaries" and "Unstable".
  • Test that the Kinect driver and OpenNI are installed properly by running the NiViewer which is in Platform/Linux/Bin/x86-Release/ under the OpenNI directory. If you get a "failed to set USB inferface" error, you need to remove the gspca_kinect kernel module by adding the following line to the file /etc/modprobe.d/blacklist.conf.

    blacklist gspcs_kinect  

    You need sudo permission to edit that file.

Jar files

Put the following jar files in the lib directory:

Other dependencies:

How to build and run

  • To build all the code, type ant build at the command under the main project directory.
  • The main class for hand tracking is edu.mit.yingyin.tabletop.apps.HandTrackingApp. To run it, type ant HandTrackingApp. The program assumes the config and data directories are under the main project direcotry.
  • For a sample .oni file, you can download it from here, and put it in the data directory. The default configuration for OpenNI is to run from the recording file ./data/sample.oni.
  • The OpenNI config file config.xml in config folder is currently set to run from the actual sensor.
  • For more information, please refer to the wiki page.