Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Hand tracking on a tabletop display using Kinect
Java Ruby
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
config
data/calibration Fixed a bug of NullPointerException.
lib
ruby Added depth value visualization.
src/edu/mit/yingyin
test/edu/mit/yingyin Added plotting of clusters.
test_data
.classpath
.gitignore Switch back to use findResource to create scheduler.
.project Removed scala nature.
README.md
Rakefile
build.xml
makefile

README.md

This code is part of the research project for gestural input on a tabletop display. One Microsoft's Kinect is used for tracking hands. The Kinect is placed above the tabletop about 1.2m above. This picture shows the setup. Currently, the code is only tested with this setup. Certain parameters are set based on the assumptions according to this setup, e.g., the distance of the tabletop from the Kinect. A different setup may cause some issues.

Environment

  • Currently developing on Ubuntu 12.04 with Sun Java 7. Tested on Windows 7.

Prerequisites

Install OpenNI framework and the corresponding Kinect driver

  • OpenNI (Version 1.5.4.0 unstable version - May 7th 2012)
  • SensorKinect PrimeSense Sensor Module for OpenNI (VVersion 5.1.2.1 unstable version - May 15th 2012) Kinect Mod:
    • Currently tested with unstable branch commit 2d139677f26c06fcd9fc8d4a7db033d70021624b
    • Follow the Linux installation instructions in README
    • Or use the binaries available at http://www.openni.org/Downloads/OpenNIModules.aspx. Choose "OpenNI Compliant Hardward Binaries" and "Unstable".
  • Test that the Kinect driver and OpenNI are installed properly by running the NiViewer which is in Platform/Linux/Bin/x86-Release/ under the OpenNI directory. If you get a "failed to set USB inferface" error, you need to remove the gspca_kinect kernel module by adding the following line to the file /etc/modprobe.d/blacklist.conf.

    blacklist gspcs_kinect  
    

    You need sudo permission to edit that file.

Jar files

Put the following jar files in the lib directory:

Other dependencies:

  • OpenCV 2.4.3

    • For compilation instruction, follow this guide.
    • There are a few changes needed to install all the necessary libaries:

      sudo apt-get install libavcodec-dev libavcodec53 libavformat53 libavformat-dev
      sudo apt-get install libtbb2 libtbb-dev
      

How to build and run

  • To build all the code, type ant build at the command under the main project directory.
  • The main class for hand tracking is edu.mit.yingyin.tabletop.apps.HandTrackingApp. To run it, type ant HandTrackingApp. The program assumes the config and data directories are under the main project direcotry.
  • For a sample .oni file, you can download it from here, and put it in the data directory. The default configuration for OpenNI is to run from the recording file ./data/sample.oni.
  • The OpenNI config file config.xml in config folder is currently set to run from the actual sensor.
  • For more information, please refer to the wiki page.
Something went wrong with that request. Please try again.