Skip to content

Home

Jump to bottom
diablodale edited this page Aug 8, 2011 · 30 revisions

##Setup# For Max/Jitter use, requirements are:

  • Windows (7 is recommended)
  • Max/MSP/Jitter 5.1.8 (recommended)
  • OpenNI 1.3.2.3 for Win32
  • PrimeSense NITE 1.4.1.2 for Win32
  • SensorKinect Win32 Device Driver v0.7 (based on 5.0.3.4 Primesense driver)

C/C++ developers, I additionally recommend:

  • Windows 7 64-bit Service Pack 1
  • Microsoft Visual C++ 2010 Express Version 10.0.30319.1 RTMRel
  • Hotfix for Microsoft Visual C++ 2010 Express - ENU (KB2455033) KB2455033

#Features#

  • Configuration of OpenNI by an OpenNI XML configuration file; see OpenNI documentation for format
  • ImageMap of RGB24 output in a 4-plane char jitter matrix
  • DepthMap output in a 1-plan long, float32, or float64 jitter matrix
  • IrMap output in a 1-plan long, float32, or float64 jitter matrix
  • UserPixelMap output in a 1-plan long, float32, or float64 jitter matrix
  • User events (e.g. user seen, user lost, etc.)
  • Center of mass for identified users
  • Skeleton joints with optional orientations
  • User events, center of mass, and skeleton joints in an native OSC format, max route friendly format, OSCeleton legacy format (as of 2011-07-25 OSCeleton codebase)
  • Attributes to filtering data based on position or orientation confidence, display or not the orientation data, and smooth skeleton data using OpenNI’s smoothing API
  • Compiled as a Win32 max/jitter external

##Max message support# This object supports the normal compliment of Max/Jitter messages to set/query attributes, get a summary via dumpout, etc. Its dumpout output will follow standard conventions through use of standard Max/Jitter APIs.

A bang sent to the inlet causes OpenNI to be queried for any updated data with no waiting/blocking and after that matrices, user events, user center of mass, and user skeletons to be output from the outlets.

This object also supports the message read. The message alone will cause Max to prompt the user for the location of the OpenNI XML configuration file. You may pass an optional filename as an argument to the message which will instruct this object to load it.

read myconfigfile.xml                      <--- example message sent to jit.openni inlet
read myconfigfile.xml 1                    <--- example message sent out of dumpout outlet

The read message is modeled after the jit.qt external's dumpout output format.

read filename success
    filename: the string "<none>" without the quotes or the filename jit.openni attempted to load
    success: an integer where 0 = file not loaded and 1 = file successfully loaded

You must read an OpenNI XML configuration file into the jit.openni object before any OpenNI functionality will be exposed.

##Attributes#

  • position_confidence_filter has a value of [0.0 - 1.0]. Any OSC data having a position confidence greater than or equal to this filter will be output. Note that both confidence filters must pass before output.
  • orient_confidence_filter has a value of [0.0 - 1.0]. Any OSC data having a orientation confidence greater than or equal to this filter will be output. Note that both confidence filters must pass before output.
  • output_skeleton_orientation has a value of [0, 1]. 0 is the default and means to not output orientation data. 1 means to output orientation data.
  • skeleton_smooth_factor has a value of [0.0 - 1.0]. 0.0 is the default and means no smoothing. 1.0 means infinite smoothing. Between 0.0 and 1.0 the behavior is generator dependent.
  • output_depthmap has a value of [0,1]. 1 is the default and enables output of depthmap matrices on the depthmap outlet
  • output_imagemap has a value of [0,1]. 1 is the default and enables output of imagemap matrices on the imagemap outlet
  • output_irmap has a value of [0,1]. 1 is the default and enables output of IRmap matrices on the irmap outlet
  • output_usermap has a value of [0,1]. 1 is the default and enables output of user pixel map matrices on the usermap outlet
  • output_skeleton has a value of [0,1]. 1 is the default and enables output of skeleton joints on the skeleton outlet
  • skeleton_format has a value of [0..2]. 0 is the default and means the native jit.openni OSC format, 1 is a native format that is friendly to max's route object, 2 is the OSCeleton legacy format
  • skeleton_value_type has a value of [0..2]. 0 is the default and outputs native OpenNI values (same as OSCeleton -xr), 1 outputs the native OpenNI values with projective coordinates, 2 outputs the OSCeleton's default behavior legacy "normalized" values

##User and Skeleton output#

###User Events# User events are generated by any user generator configured in XML. Events are a message or OSC method with only one level followed by an int32 which represents a OpenNI userID. The eight events currently output via the three formats are:

native OSC                 max route                OSCeleton legacy
--------------------------------------------------------------------------
/new_user userID           new_user userID          /new_user userID
/lost_user userID          lost_user userID         /lost_user userID
/calib_pose userID         calib_pose userID        /pose_detected userID
/calib_start userID        calib_start userID       /calibration_started userID
/calib_success userID      calib_success userID     /new_skel userID
/calib_fail userID         calib_fail userID        /calibration_failed userID
/exit_user userID          exit_user userID         /exit_user userID
/reenter_user userID       reenter_user userID      /reenter_user userID

Example native OSC output:

/new_user 2
/calib_pose 2
/calib_start 2

###User Center of Mass Data# User center of mass data is generated by any user generator configured in XML. Once a user is identified by the generator, you will begin receiving center of mass messages. As wiki this was written, the Primesense NITE user generator generates erratic data when it looses the user; often values of (0, 0, 0) appear. The /user_exit and /user_reenter events may aid in detection of bad data but unfortunately they are also unreliable.

Examples of output given value of skeleton_format attribute:
native OSC                 max route                OSCeleton legacy
--------------------------------------------------------------------------
/user/userID x y z         user userID x y z        /user/userID x y z
  • userid is an int32 which represents a OpenNI userID, the same userID as in the above events
  • x, y, z are float32 and their values are defined by the skeleton_value_type attribute

Example native OSC output:

/user/2 42.1323 -100.8237 1984.348

###Skeleton Joint Data# Skeleton join data is generated by any user generator with skeleton capability configured in XML. This jit.openni object currently outputs all OpenNI declared skeleton joint types for a tracked skeleton. By default, this object outputs joint position and orientation. The latter can be turned off using the output_skeleton_orientation attribute.

The user generator from PrimeSense called NITE as of 8 August 2011 has only three possible confidence values [0.0, 0.5, 1.0]. If you filter out confidence=0.5 values, it is equivalent to turning off their skeleton heuristics. You can do this in the XML files to save some CPU cycles using the following

<Node type="User">
    <Configuration>
        <Property type="int" name="SkeletonHeuristics" value="0"/>
    </Configuration>
</Node>

The format of the native OSC message skeleton_format=0 with orientation data is:

/skel/userid/jointname x y z confidPosition Xx Xy Xz Yx Yy Yz Zx Zy Zz orientPosition

The format of the native max route friendly message skeleton_format=1 with orientation data is:

skel userid jointname x y z confidPosition Xx Xy Xz Yx Yy Yz Zx Zy Zz orientPosition

The format of the OSCeleton legacy message skeleton_format=2:

/joint jointname userid x y z
  • userid is an int32 which represents a OpenNI userID, the same userID as in the above events
  • jointname is one of the strings listed below representing one of the declared OpenNI skeleton joint types
  • All numbers except userid are float32 and are are defined by the skeleton_value_type attribute.

Supported jointname for all output formats are:

head    neck    torso    waist
l_collar    l_shoulder    l_elbow    l_wrist    l_hand    l_fingertip
r_collar    r_shoulder    r_elbow    r_wrist    r_hand    r_fingertip
l_hip    l_knee    l_ankle    l_foot
r_hip    r_knee    r_ankle    r_foot

Example native OSC output skeleton_format=0 for a joint:

/skel/2/r_shoulder 1200.4 3050.3 4500.6 0.895

Example native OSC output skeleton_format=0 for a joint + orientation data:

/skel/2/r_shoulder 1200.4 3050.3 4500.6 0.895 1200.4 3050.3 4500.6 1200.4 3050.3 4500.6 1200.4 3050.3 4500.6 0.500
Clone this wiki locally