DollarUnity

A Unity-specific implementation of the "$1 Recognizer"

The "$1 Recognizer" is a well-known gesture recognizer developed by Jacob Wobbrock, Andy Wilson and Yang Li. DollarUnity is an adaptation of their original code to run well in Unity3d projects. Though designed for identifying pen gestures on mobile devices, the $1 Recognizer is actually able to match patterns in any stream of data, provided some preconditions are met:

• The stream must have a beginning and an end. Dollar is efficient, but it's definitely not trivial, so you don't want to run it every frame. Also, different users will complete gestures in different amounts of time, so unless the user has a way to mark the start and finish, you'd wind up having to run multiple comparisons at once (e.g., testing the last half-second, the last second and the last two seconds of data). For pen and touch gestures, the start is simply when the pen or finger makes contact, and the end is when the contact is broken. For other contexts, you might need to give the user a button to hold down throughout the gesture.
• The stream must be continuous. Dollar is a "unistroke recognizer" ("$1" is a playful abbreviation of "unistroke"), which means that there must be no discontinuities in the gesture. It's not a good choice for block letters, which are usually composed of multiple strokes. It can recognize most cursive letters just fine, but it can't distinguish a lower-case I from a lower-case T.
• Every gesture produces a match. Dollar works by finding the closest match between the user's input and a pre-defined library of gesture candidates. It thus always reports back a match, even when that "match" is not very close at all. This is another reason why it's important for the user to communicate the intent to make a gesture.

Usage

1. Create an instance of the DollarRecognizer class. This is a standard C# class, not a MonoBehaviour, so just instantiate it normally as part of one of your components.
2. Initialize the pattern library. A pattern consists of an arbitrary stream of Vector2 values. You can load this stream any way you like, then pass it to SavePattern(string name, IEnumerable<Vector2> points). The "name" value is the identifier for this gesture. You can provide multiple patterns for a given gesture, to handle variations. For example, if you want to recognize both clockwise and counter-clockwise circles as the same gesture, you'd provide two variations. Generally a single version of any gesture is sufficient.
3. When the user completes a gesture, recognize it. Pass the points to Recognize(IEnumerable<Vector2> points) to get back a Result object. This object gives you the gesture (as Match.Name), the strength of the match (as Score; higher values are better) and the angle (as Angle, in degrees) relative to the original pattern. That angle value lets you recognize an arrow gesture, for example, and determine which direction the arrow is pointing.

Backlog

• This particular implementation is poor at handling swipes. A "swipe" is just a straight line. The original $1 Recognizer is notoriously poor at handling gestures that are essentially one-dimensional, because the algorithm scales all gestures to fit within the same 2D bounding box; doing so greatly amplifies any noise in 1D gestures. The code as currently implemented here uses Yang Li's subsequent ["Protractor" refinement[(http://yangl.org/pdf/protractor-chi2010.pdf) of the $1 Recognizer, but the bounding-box scaling is still present. In theory, simply commenting out line 113 of DollarRecognizer.cs will solve this, but I haven't spent the time to confirm and debug that yet.
• Make the code coroutine-friendly. Right now the entire recognition operation is done synchronously, which is of course a poor way to do lengthy, computation-heavy tasks. It'd be easy to create an IEnumerator-returning coroutine version of the code. Note that DollarRecognizer isn't a MonoBehaviour, so you can't actually call StartCoroutine() from within it, but it can have methods that are themselves invoked by coroutines.