Skip to content

Home

Jump to bottom
Dale Phurrough edited this page May 14, 2014 · 30 revisions

##Setup# Operating Systems

  • Modern Windows 32-bit compatible OS, e.g. Windows 7 (32 or 64 bit)
  • Mac OSx 10.6.8 or newer

Application Environment/Platform
jit.openni is an extension (an enhancement) for the Max application platform. Max is created by Cycling 74 and allows you to create your own music, sound, video, and interactive media applications. More information at http://cycling74.com.

Download and Install
Download the needed software for your OS from http://hidale.com/jit-openni/#download
Install them in the order listed:

  • Only for Mac OSx: libusb-1.0.9
  • OpenNI 1.5.4.0
  • PrimeSense NITE 1.5.2.21
  • PrimeSense Sensor KinectMod v0.93 (based on 5.1.2.1)
  • Max 6.0.7 or higher for Windows (earlier versions may also work)

Additional Windows install notes

  • Windows 8 and newer versions require driver signatures. The author of the Sensor KinectMod did not sign it. Therefore, Windows 8 will not install the driver by default. There are some workarounds that might work which temporarily disable Windows 8 driver signature enforcement.

Additional Mac OSx install notes

  • expand, run, and install the libusb-1.0.9 package
  • expand OpenNI and in that folder sudo ./install.sh
  • expand NITE and in that folder sudo ./install.sh
  • expand SensorKinect and in that folder sudo ./install.sh

##Features#

  • Configuration of OpenNI by an OpenNI XML configuration file; see OpenNI documentation for format
  • ImageMap of RGB24 output in a 4-plane char jitter matrix
  • DepthMap output in a 1-plan long, float32, or float64 jitter matrix
  • IrMap output in a 1-plan long, float32, or float64 jitter matrix
  • UserPixelMap output in a 1-plan long, float32, or float64 jitter matrix
  • User events (e.g. user seen, user lost, etc.)
  • Center of mass for identified users
  • Floor identification
  • Skeleton joints with optional orientations
  • User events, center of mass, and skeleton joints in an native OSC format, max route friendly format, or OSCeleton legacy format (as of 2011-07-25 OSCeleton codebase)
  • Attributes to filtering data based on position or orientation confidence, display or not the orientation data, and smooth skeleton data using OpenNI’s smoothing API
  • Camera field of view
  • Compiled as a Win32 max/jitter external

##Max message support# This object supports the normal compliment of Max/Jitter messages to set/query attributes, get a summary via dumpout, etc. Its dumpout output will follow standard conventions through use of standard Max/Jitter APIs.

bang sent to the inlet causes OpenNI to be queried for any updated data with no waiting/blocking and after that matrices, user events, user center of mass, and user skeletons to be output from the outlets.

read the message alone will cause Max to prompt the user for the location of the OpenNI XML configuration file. You may pass an optional filename as an argument to the message which will instruct this object to load it.

read myconfigfile.xml                      <--- example message sent to jit.openni inlet
read myconfigfile.xml 1                    <--- example message sent out of dumpout outlet

The read message is modeled after the jit.qt external's dumpout output format.

read filename success
    filename: the string "<none>" without the quotes or the filename jit.openni attempted to load
    success: an integer where 0 = file not loaded and 1 = file successfully loaded

You must read an OpenNI XML configuration file into the jit.openni object before any OpenNI functionality will be exposed.

getversions sends the version numbers to the dumpout for jit.openni and OpenNI. Example output is:

version_jit.openni 0 7 7
version_openni 1 5 4 0

##Attributes#

  • distmeter has a value of [0,1] where the default 0 provides distances in millimeters and a value of 1 provides distances in meters
  • position_confidence_filter has a value of [0.0 - 1.0]. Any OSC data having a position confidence greater than or equal to this filter will be output. Note that both confidence filters must pass before output.
  • orient_confidence_filter has a value of [0.0 - 1.0]. Any OSC data having a orientation confidence greater than or equal to this filter will be output. Note that both confidence filters must pass before output.
  • output_skeleton_orientation has a value of [0, 1]. 0 is the default and means to not output orientation data. 1 means to output orientation data.
  • skeleton_smooth_factor has a value of [0.0 - 1.0]. 0.0 is the default and means no smoothing. 1.0 means infinite smoothing. Between 0.0 and 1.0 the behavior is generator dependent.
  • output_depthmap has a value of [0,1]. 1 is the default and enables output of depthmap matrices on the depthmap outlet; depth is in millimeters
  • output_imagemap has a value of [0,1]. 1 is the default and enables output of imagemap matrices on the imagemap outlet
  • output_irmap has a value of [0,1]. 1 is the default and enables output of IRmap matrices on the irmap outlet
  • output_usermap has a value of [0,1]. 1 is the default and enables output of user pixel map matrices on the usermap outlet
  • output_skeleton has a value of [0,1]. 1 is the default and enables output of skeleton joints on the skeleton outlet
  • output_scene_floor has a value of [0,1]. 0 is the default and disables scene floor data being output on the OSC outlet for each bang. 1 enables floor data on the OSC outlet. In both cases, the floor data is available on the scene_floor attribute.
  • scene_floor is an array of 6 floats: first 3 are the X, Y, Z coordinates for a point on the plane, the second 3 are the X, Y, Z of a normal vector. These are the same values as are output with the floor OSC message
  • skeleton_format has a value of [0..2]. 0 is the default and means the native jit.openni OSC format, 1 is a native format that is friendly to max's route object, 2 is the OSCeleton legacy format
  • skeleton_value_type has a value of [0..2]. 0 is the default and outputs native OpenNI values in millimeters (same as OSCeleton -xr), 1 outputs the native OpenNI values with projective coordinates, 2 outputs the OSCeleton's default behavior legacy "normalized" values
  • depth_fov is an array of 2 floats representing the depth camera's horizontal and vertical (in that order) field of view in radians

##User and Skeleton output#

###User Events# User events are generated by any user generator configured in XML. Events are a message or OSC method with only one level followed by an int32 which represents a OpenNI userID. The eight events currently output via the three formats are:

native OSC                 max route                OSCeleton legacy
--------------------------------------------------------------------------
/new_user userID           new_user userID          /new_user userID
/lost_user userID          lost_user userID         /lost_user userID
/calib_pose userID         calib_pose userID        /pose_detected userID
/calib_start userID        calib_start userID       /calibration_started userID
/calib_success userID      calib_success userID     /new_skel userID
/calib_fail userID         calib_fail userID        /calibration_failed userID
/exit_user userID          exit_user userID         /exit_user userID
/reenter_user userID       reenter_user userID      /reenter_user userID

Example native OSC output:

/new_user 2
/calib_pose 2
/calib_start 2

###User Center of Mass Data# User center of mass data is generated by any user generator configured in XML. Once a user is identified by the generator, you will begin receiving center of mass messages. As wiki this was written, the Primesense NITE user generator generates erratic data when it looses the user; often values of (0, 0, 0) appear. The /user_exit and /user_reenter events may aid in detection of bad data but unfortunately they are also unreliable.

Examples of output given value of skeleton_format attribute:
native OSC                 max route                OSCeleton legacy
--------------------------------------------------------------------------
/user/userID x y z         user userID x y z        /user/userID x y z
  • userid is an int32 which represents a OpenNI userID, the same userID as in the above events
  • x, y, z are float32 and their values are defined by the skeleton_value_type attribute

Example native OSC output:

/user/2 42.1323 -100.8237 1984.348

###Scene Floor Data# Scene floor idenfication is generated by any scene generator configured in XML. Once a floor is identified by the generator, you will begin receiving floor messages if configured using the output_scene_floor attribute. When this wiki was written, the Primesense NITE scene generator outputs data that describes a floor plane relative to the sensor's 0,0,0 origin. The floor point's X value is always 0.0, Z appears to stablize at 349.18811, and the remaining values change based on your physical setup.

Remember that this same floor data is available in the scene_floor attribute and can be queried using standard Max messages like "getscene_floor". When done, its results will be output via dumpout outlet. The x, y, z coordinates are by default in millimeters.

Examples of output given value of skeleton_format attribute:
native OSC                 max route                OSCeleton legacy
--------------------------------------------------------------------------
/floor x y z xn yn zn      floor x y z xn yn zn     /floor x y z xn yn zn
  • x, y, z are float32 and are coordinates for a point on the floor plane
  • xn, yn, zn are float32 and are of a vector normal to the floor plane

Example native OSC output:

/floor 0.0 -925.784 349.18811 0.128 0.9721 0.0472

###Skeleton Joint Data# Skeleton join data is generated by any user generator with skeleton capability configured in XML. This jit.openni object currently outputs all OpenNI declared skeleton joint types for a tracked skeleton. By default, this object outputs joint position (mm) and orientation. You can output in meters or add/remove the orientation data with the distmeter and output_skeleton_orientation attributes.

The orientation data is output as 9 floats as a 3x3 rotation (orthonormal) matrix. This represents a rotation between the joint's local coordinates and the world coordinates. The first three are the direction of the joint's +X axis given as a 3D vector, the second three are the +Y axis direction, and the third three are the +Z axis direction. +X points to the right, +Y points up, and +Z points in the direction of increasing depth. All are in the world coordinate system. The un-rotated neutral pose is a "T pose". This is where the human stands straight upright facing the sensor; both arms parallel to the group and fully extending to the left/right; and the upper arm is twisted such that if the elbow is flexed, the lower arm will bend towards the sensor. In this neutral pose, each joint's orientation is aligned with the world coordinate system. That is, its orientation is the identity matrix (1,0,0,0,1,0,0,0,1).

The user generator from PrimeSense called NITE as of 8 August 2011 implements only 15 of the possible 24 OpenNI joints. In addition, NITE has only three possible confidence values [0.0, 0.5, 1.0]. If you filter out confidence=0.5 values, it is equivalent to turning off their skeleton heuristics. You can do this in the XML files to save some CPU cycles using the following

<Node type="User">
    <Configuration>
        <Property type="int" name="SkeletonHeuristics" value="0"/>
    </Configuration>
</Node>

The format of the native OSC message skeleton_format=0 with orientation data is:

/skel/userid/jointname x y z confidPosition Xx Xy Xz Yx Yy Yz Zx Zy Zz confidOrient

The format of the native max route friendly message skeleton_format=1 with orientation data is:

skel userid jointname x y z confidPosition Xx Xy Xz Yx Yy Yz Zx Zy Zz confidOrient

The format of the OSCeleton legacy message skeleton_format=2:

/joint jointname userid x y z
  • userid is an int32 which represents a OpenNI userID, the same userID as in the above events
  • jointname is one of the strings listed below representing one of the declared OpenNI skeleton joint types
  • All numbers except userid are float32 and are are defined by the skeleton_value_type attribute.

Supported jointname for all output formats are:

head    neck    torso    waist
l_collar    l_shoulder    l_elbow    l_wrist    l_hand    l_fingertip
r_collar    r_shoulder    r_elbow    r_wrist    r_hand    r_fingertip
l_hip    l_knee    l_ankle    l_foot
r_hip    r_knee    r_ankle    r_foot

Example native OSC output skeleton_format=0 for a joint:

/skel/2/r_shoulder 1200.4 3050.3 4500.6 0.895

Example native OSC output skeleton_format=0 for a joint + orientation data:

/skel/2/r_shoulder 1200.4 3050.3 4500.6 0.895 1200.4 3050.3 4500.6 1200.4 3050.3 4500.6 1200.4 3050.3 4500.6 0.500
Clone this wiki locally