TUIO proxy

The proxy software is the program that receives the signals emitted by the infrared overlay’s parsing software. In those messages it looks for tangible patterns and, if recognised, sends messages with position and rotation of the tangibles to the client software.

#What it does

##Normal message flow

When a user touches the screen the infrared overlay detects the disruptions in its infrared grid. The data generated by the frame is sent via USB to the parsing software of the overlay vendor. This software creates special network messages with the position of the touches which are then received by the software with the user interface.

Figure 41. Usual data flow for multi-touch interactions

##Message flow with proxy

A new program is inserted in the data stream. It takes a look into all the cursor messages sent from the parsing software.

If it detects a known pattern of cursors it inserts additional message in the cursor stream. Those “object messages” contain the ID, position and rotation of the tangible.

Figure 42. The multi-touch data flow through the proxy

#How it works in detail

The proxy software sends and receives TUIO messages. TUIO is based on Open Sound Control, a protocol created to function as interface between multimedia devices and software. [Wright, M., Freed, A., Momeni A. (2003)].

##TUIO protocol

Basically the TUIO protocol transports information about tangible positions, rotations and of course simple touches towards applications that then react to those inputs. One can consider it as mouse or finger positions transferred over a network protocol.

To group and specify what is going on the touch-surface several message profiles are defined. The messages mainly contain a position and acceleration.

For our purpose two message profiles are used:

• 2DCur - A two dimensional cursor (e.g. a finger touch)
• 2DObj - Same as a 2DCur, but with more information like rotation and ID

A 2DCur message looks like this:

/tuio/2Dcur set s x y X Y m

• s = session ID

• x,y = position

• X,Y = velocity vector

• m = motion acceleration

A 2DObj message looks like this:

/tuio/2Dobj set s i x y a X Y A m r

• s = session ID

• i = class ID

• x,y = position

• a = angle

• X,Y = velocity vector

• A = rotation velocity vector

• m = motion acceleration

• r = rotation acceleration

Both message types are by standard bundled with a previous alive message giving the IDs of the cursors that are still active. The message looks like this:

/tuio/2Dcur alive s_id0 ... s_idN

At the end a message with an incrementing sequence number is attached:

/tuio/2Dcur fseq f_id

In total a message bundle would look like this:

/tuio/2Dcur alive s_id0 ... s_idN

/tuio/2Dcur set s_id x_pos y_pos x_vel y_vel m_accel

/tuio/2Dcur fseq f_id

For further information, please visit TUIO.org where also the previous specification examples are taken from.

#The recognition and calculation

The recognition and calculation basically follows the scheme described in the algorithm chapter.

The following diagrams show the significant code parts as UML diagrams.

Figure 43. Schematic overview of the TUIO message handling

Figure 44. Creating all possible triangles from the received cursors

Figure 45. v1-v2 the short, v2-v3 the intermediate, v3-v1 the long side

Figure 46. Compare tangible triangles and triangles of momentary

Figure 47. Checking two triangles for similarity

Figure 48. Calculate the tangible position and rotation after detection

Figure 49. Remove points belonging to recognized tangibles from 2Dcur buffer