Skip to content

Writing a Custom Gesture

Valentin Simonov edited this page Aug 2, 2017 · 1 revision

In TouchScript only gestures can receive touch events. The library comes with several built-in gestures which handle most of the cases, but sometimes it's just easier to write a new one than try to combine them. This knowledge also is useful for better understanding how TouchScript works.

Read more about gestures and gesture types.

This tutorial assumes that the reader knows C#.

The task

Let's imagine that we are making a game where we have a 3D object which we can shoot in a sense of Angry Birds game, by dragging a ribbon out of it and releasing — the farther we drag the ribbon, the larger the force shooting the object is.

We can approach this task in a couple ways using built-in gestures. For example, have a PressGesture on the object which then spawns an invisible dummy with a collider and cancels itself so the dummy would catch the touch. The dummy will have TranformGesture and Transformer components attached so we could drag it around. The dummy will work as our end of the ribbon. When TransformGesture finishes we calculate the distance between the object and the dummy to apply the force.

This approach sounds complicated and is a great candidate for a custom gesture.

What should this gesture do?

Now we need to think about what the new gesture is supposed to do exactly.

We want it to behave the following way:

  1. It should care only about the first active touch.
  2. It should be continuous and dispatch events when dragged so we could update our logic.
  3. When the object is pressed it should fire an event and start.
  4. When released it should fire an event.

Let's get to the code

We will start with the table and a box used in example scenes. This is the box we want to "kick" using the gesture we are going to write. The box must have BoxCollider and RigidBody components to be able to work with Unity physics.

Next, let's create a C# script, name it PullGesture and add this code inside:

using UnityEngine;
// Must import this to use Gesture type
using TouchScript.Gestures;

// Let's put our gesture into a namespace so it wouldn't clash with other classes in our project
namespace TouchScript.Tutorial
{
    // The class must inherit from Gesture
    public class PullGesture : Gesture 
    {
    }
}

Let's attach this gesture to the Box but it is not doing anything useful right now. To make the gesture "listen" to pointer events we need to add the following code to the class:

// Pointers pressed this frame
protected override void pointersPressed(IList<Pointer> pointers)
{
    Debug.Log("Pressed!");
}

// Pointers updated this frame
protected override void pointersUpdated(IList<Pointer> pointers)
{
    Debug.Log("Updated!");
}

// Pointers released this frame
protected override void pointersReleased(IList<Pointer> pointers)
{
    Debug.Log("Released!");
}

// Pointers cancelled this frame
protected override void pointersCancelled(IList<Pointer> pointers)
{
    Debug.Log("Cancelled!");
}

At this point your IDE will complain about missing classes, so just add the following usings on top of the file:

using System.Collections.Generic;
using TouchScript.Pointers;

pointersPressed, pointersUpdated, pointersReleased and pointersCancelled methods are called when the gesture receives pointer events, and here is the place where the logic looking for patterns should be implemented. But, first let's add events other scripts expect from this gesture.

Note: it is not mandatory to have custom events in a gesture, other scripts can always subscribe to StateChanged event and listen to the gesture state changes, but having custom dedicated events is always nicer.

public event EventHandler<EventArgs> Pressed
{
    add { pressedInvoker += value; }
    remove { pressedInvoker -= value; }
}

public event EventHandler<EventArgs> Pulled
{
    add { pulledInvoker += value; }
    remove { pulledInvoker -= value; }
}

public event EventHandler<EventArgs> Released
{
    add { releasedInvoker += value; }
    remove { releasedInvoker -= value; }
}

// Needed to overcome iOS AOT limitations
private EventHandler<EventArgs> pressedInvoker, pulledInvoker, releasedInvoker;

At this point you will also need to add this using to be able to use EventHandler and EventArgs classes:

using System;

Let's add the first event

Now that we have our nice events, let's write the actual logic behind the gesture.

First, let's update pointersPressed method. We want our gesture to start immediately when a pointer is pressed.

protected override void pointersPressed(IList<Pointer> pointers)
{
    if (State == GestureState.Idle)
    {
        primaryPointer = pointers[0];
        // Start the gesture
        setState(GestureState.Began);
    }
}

You will also need to add this using to the top of the script file:

using TouchScript.Layers;

And definition of the private variable we are setting:

// The only pointer we are interested in
private Pointer primaryPointer;

This gesture is interested only in the first pointer and will work only with it. This is why we want to save it for future reference. The code above basically says: "If we are in Idle state, save the pointer and start".

At this point we want to dispatch Pressed event, but it is better to do this in the following method (add this code to your class):

// Called when the gesture transitions to Began state
protected override void onBegan()
{
    if (pressedInvoker != null) pressedInvoker(this, EventArgs.Empty);
}

onIdle, onPossible, onBegan, onChanged, onRecognized, onFailed and onCancelled methods are called when gesture transitions in the corresponding state and are the places in the code where you should add logic for the state instead of having this logic in pointersPressed, for example.

Let's add other events

Now let's add the code to update and finish the gesture:

// Pointers updated this frame
protected override void pointersUpdated(IList<Pointer> pointers)
{
    foreach (var p in pointers)
    {
        if (p.Id == primaryPointer.Id) 
        {
            // If the pointer we are interested in moved, change the state
            setState(GestureState.Changed);
            return;
        }
    }
}

// Pointers released this frame
protected override void pointersReleased(IList<Pointer> pointers)
{
    foreach (var p in pointers)
    {
        if (p.Id == primaryPointer.Id) 
        {
            // If the pointer we are interested was released, end the gesture
            setState(GestureState.Ended);
            return;
        }
    }
}

// Pointers cancelled this frame
protected override void pointersCancelled(IList<Pointer> pointers)
{
    foreach (var p in pointers)
    {
        if (p.Id == primaryPointer.Id) 
        {
            // If the pointer we are interested was cancelled, cancel the gesture
            setState(GestureState.Cancelled);
            return;
        }
    }
}

// Called when the gesture transitions to Ended or Recognized states
protected override void onRecognized()
{
    if (releasedInvoker != null) releasedInvoker(this, EventArgs.Empty);
}

// Called when the gesture transitions to Changed state
protected override void onChanged()
{
    if (pulledInvoker != null) pulledInvoker(this, EventArgs.Empty);
}

The code in pointersUpdated only updates the gesture if the first pointer is moved. The code in pointersReleased says that if we released the first pointer we finish the gesture. The same for pointersCancelled but in this case we cancel the gesture.

Note: the difference between finishing and cancelling a gesture is that when cancelled the gesture must not perform the action it does when finished. In our case it is kicking the object.
A gesture and a pointer can be cancelled by the system or using special API.

And last, we need to reset some values when the gesture returns to Idle state. This is done by overriding reset method like so:

// This method is called when gesture is reset when recognized or failed
protected override void reset()
{
    base.reset();
    primaryPointer = null;
}

Now we need to return correct data

Alright, we have a gesture which dispatches a few events during its lifecycle, but we also need some data from the gesture to be able to use it:

  1. 3D position of the point where the user pressed the cube,
  2. Current 3D position of the dragged pointer.

To provide accurate data we will need a few things. First, we will need correct plane where we want to project pointers from screen. Add this code to the class:

// 3D plane to project to
private Plane plane;

// Layer projection parameters
private ProjectionParams projection;

// The world coordinates of the point where the gesture started
private Vector3 startPosition;

projection variable stores layer projection settings which we will use later to project pointer screen position into 3D world. Layers can have different projection parameters depending on Camera settings and whatnot, but generally if the first pointer comes from a certain layer, others will come from the same layer.

Also update pointersPressed method to look like so:

protected override void pointersPressed(IList<Pointer> pointers)
{
    if (State == GestureState.Idle)
    {
        primaryPointer = pointers[0];
        projection = primaryPointer.GetPressData().Layer.GetProjectionParams(primaryPointer);
        plane = new Plane(Vector3.up, transform.position);
        startPosition = projection.ProjectTo(primaryPointer.Position, plane);

        // Start the gesture
        setState(GestureState.Began);
    }
}

Here we are setting projection and plane to use while the gesture is active.

Now let's add public properties to return start, current positions and current force.

public Vector3 StartPosition
{
    get 
    {
        switch (State)
        {
            case GestureState.Began:
            case GestureState.Changed:
            case GestureState.Ended:
                return startPosition;
            default:
                return transform.position;
        }
    }
}

public Vector3 Position
{
    get
    {
        switch (State)
        {
            case GestureState.Began:
            case GestureState.Changed:
            case GestureState.Ended:
                return projection.ProjectTo(primaryPointer.Position, plane);
            default:
                return transform.position;
        }
    }
}

public Vector3 Force
{
    get 
    {
        return StartPosition - Position;
    }
}

Notice how we need to check in what state the gesture currently is, because these values don't make sense when the gesture is not active. You can add the following code to onChanged method to test that the properties actually work:

Debug.LogFormat("Start position: {0}, current position: {1}, force: {2}", StartPosition, Position, Force.magnitude);

See the final version of PullGesture.cs here.

Let's add the script which controls the Box

The gesture we just wrote handles pointers and returns data we need. Now let's write the script which actually uses and applies this data.

Create a new C# script, name it Logic and add the following code into it:

using UnityEngine;

namespace TouchScript.Tutorial
{
    public class Logic : MonoBehaviour 
    {
        private PullGesture gesture;
        private Rigidbody body;

        private void OnEnable()
        {
            body = GetComponent<Rigidbody>();
            gesture = GetComponent<PullGesture>();

            gesture.Pressed += pressedHandler;
            gesture.Pulled += pulledHandler;
            gesture.Released += releasedHandler;

            releaseObject();
        }

        private void OnDisable()
        {
            gesture.Pressed -= pressedHandler;
            gesture.Pulled -= pulledHandler;
            gesture.Released -= releasedHandler;
        }

        // Switch to manual mode
        private void takeObject()
        {
            body.isKinematic = true;
        }

        // Switch to automatic mode
        private void releaseObject()
        {
            body.isKinematic = false;
        }

        private void pressedHandler(object sender, System.EventArgs e)
        {
            takeObject();
            Debug.Log("Pressed!");
        }

        private void pulledHandler(object sender, System.EventArgs e)
        {
            Debug.LogFormat("Updated! Start position: {0}, current position: {1}, force: {2}", 
                gesture.StartPosition, gesture.Position, gesture.Force.magnitude);
        }

        private void releasedHandler(object sender, System.EventArgs e)
        {
            releaseObject();
            Debug.LogFormat("Released! Start position: {0}, current position: {1}, force: {2}", 
                gesture.StartPosition, gesture.Position, gesture.Force.magnitude);
        }

        private void cancelledHandler (object sender, System.EventArgs e)
        {
            releaseObject();
        }
    }
}

This code subscribes to the PullGesture's events and provides two methods to control object's behavior: takeObject and releaseObject. We will use these to control if Unity built-in physics should work with the object or not. As with PullGesture we will use our custom TouchScript.Tutorial namespace.

Now we need to write the code which adds the force. Add these variables we will need to the code:

// Force multiplier
public float ForceMultiplier = 100f;

private Vector3 forceToApply;
private bool shouldApplyForce = false;

Add these methods:

private void FixedUpdate()
{
    // Apply force in FixedUpdate to make physics happy
    if (shouldApplyForce) 
    {
        body.AddForce(forceToApply);
        shouldApplyForce = false;
    }
}

// Push the object when the gesture is ended
private void pushObject()
{
    forceToApply = ForceMultiplier * gesture.Force;
    shouldApplyForce = true;
}

We will use pushObject to set up the force and apply it in the next FixedUpdate.

Now we just need to change releasedHandler to call this method:

private void releasedHandler(object sender, System.EventArgs e)
{
    releaseObject();
    pushObject();
}

Now go to Unity, press Play and try pulling and releasing the box.

Let's add some visuals

You are probably asking why we created Pulled event if we left pulledHandler empty anyway. Let's use this event to add a visual representation of a line between the Box and current pointer.

First add an empty GameObject as a child to the Box and attach LineRenderer component to it. Set its parameters like so and attach a material (we used bright green in this tutorial).

Next, modify Logic script to update this LineRenderer when needed. Add a field, switch to the editor and drag newly created LineRenderer into it:

public LineRenderer Line;

Add this line to OnEnable to hide the LineRenderer:

private void OnEnable()
{
    ...
    Line.enabled = false;
}

Add this method to the class to update LineRenderer:

// Update the line
private void updateLine()
{
    Line.SetPosition(0, gesture.StartPosition);
    Line.SetPosition(1, gesture.Position);
}

And finally update the following methods:

// Switch to manual mode
private void takeObject()
{
    body.isKinematic = true;
    Line.enabled = true;
    updateLine();
}

// Switch to automatic mode
private void releaseObject()
{
    body.isKinematic = false;
    Line.enabled = false;
}

private void pulledHandler(object sender, System.EventArgs e)
{
    updateLine();
}

Now switch to Unity and try pulling the Box again. You should see the line from the Box to the pointer.

See the final version of Logic.cs here.

Where to go from here

Now you know how to create custom gestures. From here you should go and read the code to find out how built-in gestures and custom inspectors for them are implemented. We suggest you start from the following scripts:

You can’t perform that action at this time.