Skip to content

Writing an Input Source

Valentin Simonov edited this page Nov 7, 2017 · 3 revisions

In TouchScript all Pointers are created by Input Sources, they:

  • Create pointers,
  • Correctly set position and other data for these pointers,
  • Recycle pointers when needed.

If we have a real or virtual device which sends pointer data in a format TouchScript doesn't understand, the most correct way to use this data is to create a special Input Source which translates data from that device into TouchScript Pointers. This tutorial describes how to do just that.

Polling or event model

Before writing a custom Input Source we need to know how the device we are working with sends data and what API (if any) it has. There are two models:

  1. The device sends data as some kind of C# events.
  2. We have to ask it every frame if there is new data.

In the first case we'd just subscribe to these events and handle them accordingly. In the second case we need to poll the device manually in UpdateInput() method which gets called by TouchScript before updating pointers.

Device API

For this tutorial let's assume the following about the device:

  1. It is a touch device, so if a pointer exists, it is always in Pressed state (as a touch screen but not mouse, for example).
  2. For every pointer it gives us two values: pointer id and pointer position.
  3. It has an event based API.

This is the API that the device provides:

namespace MyDevice
{
    public class DeviceProxy 
    {
        public static DeviceProxy Instance { ... }

        public event EventHandler<DeviceProxyEventArgs> PointerPressed;
        public event EventHandler<DeviceProxyEventArgs> PointerMoved;
        public event EventHandler<DeviceProxyEventArgs> PointerReleased;
    }

    public class DeviceProxyEventArgs : EventArgs
    {
        public int Id;
        public Vector2 Position;
    }
}

Where Position is in meters.

Let's start writing the Input Source

First, we should look at IInputSource interface. It has several elements:

  1. ICoordinatesRemapper CoordinatesRemapper { get; set; } — coordinate remapper to convert pointer coordinates, this is taken care by InputSource class.
  2. bool UpdateInput() — this method is called every frame before TouchScript processes new pointers, because our device API is event-based we don't need to worry about it.
  3. void UpdateResolution() — this method has to be manually called if user changes resolution, taken care by InputSource class.
  4. bool CancelPointer(Pointer pointer, bool shouldReturn) — this method is called by the system to cancel and return a pointer, for things to work correctly we will need to implement it.
  5. void INTERNAL_DiscardPointer(Pointer pointer) — this is an internal method and users shouldn't call it directly, it is used by TouchScript to let the Input Source reuse discarded (not used by TS anymore) pointers.

Let's create a new class and call it MyInputSource, like so:

public class MyInputSource : InputSource 
{
    public float Width = 4;
    public float Height = 3;
}

We will need to specify actual device dimensions in meters to be able to convert to screen coordinates. Let's assume that our current device is 4 by 3 meters wide.

Let's add some code to see that the device actually sends data:

protected override void OnEnable()
{
    base.OnEnable();

    var device = DeviceProxy.Instance;
    if (device != null)
    {
        device.PointerPressed += pointerPressedHandler;
        device.PointerMoved += pointerMovedHandler;
        device.PointerReleased += pointerReleasedHandler;
    }
}

protected override void OnDisable()
{
    var device = DeviceProxy.Instance;
    if (device != null)
    {
        device.PointerPressed -= pointerPressedHandler;
        device.PointerMoved -= pointerMovedHandler;
        device.PointerReleased -= pointerReleasedHandler;
    }

    base.OnDisable();
}

private void pointerPressedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} added at {1}.", e.Id, e.Position);
}

private void pointerMovedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} moved to {1}.", e.Id, e.Position);
}

private void pointerReleasedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} removed.", e.Id);
}

If you don't see any messages in console when "touching" the device, you need to figure out that the device actually works before continuing.

Pointers and pooling

To represent touches and other pointer types TouchScript uses Pointer class and classes which extend it: TouchPointer, MousePointer, PenPointer and ObjectPointer. Each of these has custom fields specific for that type. Our device is a touch device so our new Input Source will feed TouchPointers into the system.

Each Input Source is responsible for creating and reusing pointers. To inject a pointer into the system, an Input Source must call addPointer() method. When a pointer is removed either by using removePointer() method or when the system cancels it, INTERNAL_DiscardPointer() method is called on the Input Source which created this pointer. At this point the Input Source should reuse the pointer object to reduce managed memory allocations.

For this purpose, TouchScript has ObjectPool<T> class. Let's add a pool into the new Input Source.

private ObjectPool<TouchPointer> touchPool;

public MyInputSource()
{
    touchPool = new ObjectPool<TouchPointer>(10, () => new TouchPointer(this), null, resetPointer);
}

private void resetPointer(Pointer p)
{
    p.INTERNAL_Reset();
}

Adding and removing pointers

Now we can actually add code for adding and removing pointers in TouchScript. First, we need to store a map from device touch id to TouchScript touch id like so:

private Dictionary<int, TouchPointer> deviceIdToTouch = new Dictionary<int, TouchPointer>(10);

Next, we need some code to add a pointer at certain screen coordinates to TouchScript:

private TouchPointer internalAddTouch(Vector2 position)
{
    // Get a pointer from the pool
    var pointer = touchPool.Get();
    // Set its (remapped) position
    pointer.Position = remapCoordinates(position);
    // Set its buttons as "button one pressed this frame"
    pointer.Buttons |= Pointer.PointerButtonState.FirstButtonDown | Pointer.PointerButtonState.FirstButtonPressed;
    // Add the pointer to the system
    addPointer(pointer);
    // Press the pointer
    pressPointer(pointer);
    return pointer;
}

And update our pressed handler:

private void pointerPressedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} added at {1}.", e.Id, e.Position);
    lock (this)
    {
        deviceIdToTouch.Add(e.Id, internalAddTouch(new Vector2(e.Position.x / Width * screenWidth, e.Position.y / Height * screenHeight)));
    }
}

Since we don't know from which thread and when the device dispatches events, we need a lock to prevent any race conditions and data corruption. Let's add code to remove pointers:

private TouchPointer internalRemoveTouch(int id)
{
    TouchPointer pointer;
    // Check if we have a pointer with such id
    if (!deviceIdToTouch.TryGetValue(id, out pointer)) return null;

    releasePointer(pointer);
    removePointer(pointer);
    return pointer;
}

private void pointerReleasedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} removed.", e.Id);
    lock (this)
    {
        internalRemoveTouch(e.Id);
        deviceIdToTouch.Remove(e.Id);
    }
}

public override void INTERNAL_DiscardPointer(Pointer pointer)
{
    touchPool.Release(pointer as TouchPointer);
}

Moving pointers

Now our Input Source can handle adding and removing pointers from the device. Let's add some code to handle moving pointers.

private void pointerMovedHandler(object sender, DeviceProxyEventArgs e) 
{
    Debug.LogFormat("Pointer {0} moved to {1}.", e.Id, e.Position);
    lock (this)
    {
        TouchPointer touch;
        if (!deviceIdToTouch.TryGetValue(e.Id, out touch)) return;

        touch.Position = remapCoordinates(new Vector2(e.Position.x / Width * screenWidth, e.Position.y / Height * screenHeight));
        updatePointer(touch);
    }
}

Cancelling pointers

TouchScript can cancel pointers either for its internal uses or by request from user with specific API. This means that the pointer is removed from the system. But sometimes it might be required to immediately add a copy of this pointer back. This is why it is important to provide an implementation of bool CancelPointer(Pointer pointer, bool shouldReturn) method in an Input Source.

public override bool CancelPointer(Pointer pointer, bool shouldReturn)
{
    base.CancelPointer(pointer, shouldReturn);

    lock (this)
    {
        int id = -1;
        foreach (var touchPoint in deviceIdToTouch)
        {
            if (touchPoint.Value.Id == pointer.Id)
            {
                id = touchPoint.Key;
                break;
            }
        }
        if (id > -1)
        {
            cancelPointer(pointer);
            if (shouldReturn)
                deviceIdToTouch[id] = internalReturnTouch(pointer as TouchPointer);
            else
                deviceIdToTouch.Remove(id);
            return true;
        }
        return false;
    }
}

private TouchPointer internalReturnTouch(TouchPointer pointer)
{
    // Get a pointer from the pool
    var newPointer = touchPool.Get();
    // Copy data from the old pointer
    newPointer.CopyFrom(pointer);
    // Set its buttons as "button one pressed this frame"
    pointer.Buttons |= Pointer.PointerButtonState.FirstButtonDown | Pointer.PointerButtonState.FirstButtonPressed;
    // Set flags
    newPointer.Flags |= Pointer.FLAG_RETURNED;
    // Add the pointer to the system
    addPointer(newPointer);
    // Press the pointer
    pressPointer(newPointer);
    return newPointer;
}

Pointer.FLAG_RETURNED is also very important and gestures like Tap Gesture will not work if it is not set up correctly.

Post Scriptum

At this point we have a working Input Source which should work nicely with TouchScript.

Here's the final source code.

using TouchScript.InputSources;
using TouchScript.Pointers;
using MyDevice;
using UnityEngine;
using TouchScript.Utils;
using System.Collections.Generic;

namespace TouchScript.Tests.IS
{
    public class MyInputSource : InputSource 
    {

        // Device touch area width in meters
        public float Width = 1;
        // Device touch area height in meters
        public float Height = 1;

        // Pool to reuse TouchPointers
        private ObjectPool<TouchPointer> touchPool;
        // Map from device touch id to TouchScript pointer
        private Dictionary<int, TouchPointer> deviceIdToTouch = new Dictionary<int, TouchPointer>(10);

        public MyInputSource()
        {
            touchPool = new ObjectPool<TouchPointer>(10, () => new TouchPointer(this), null, resetPointer);
        }

        // Cancels the pointer from the system and optionally returns it at the same position
        public override bool CancelPointer(Pointer pointer, bool shouldReturn)
        {
            base.CancelPointer(pointer, shouldReturn);

            lock (this)
            {
                int id = -1;
                foreach (var touchPoint in deviceIdToTouch)
                {
                    if (touchPoint.Value.Id == pointer.Id)
                    {
                        id = touchPoint.Key;
                        break;
                    }
                }
                if (id > -1)
                {
                    cancelPointer(pointer);
                    if (shouldReturn)
                        deviceIdToTouch[id] = internalReturnTouch(pointer as TouchPointer);
                    else
                        deviceIdToTouch.Remove(id);
                    return true;
                }
                return false;
            }
        }

        // Called by TouchScript when the pointer is no longer needed
        public override void INTERNAL_DiscardPointer(Pointer pointer)
        {
            touchPool.Release(pointer as TouchPointer);
        }

        protected override void OnEnable()
        {
            base.OnEnable();

            // Subscribe to DeviceProxy events
            var device = DeviceProxy.Instance;
            if (device != null)
            {
                device.PointerPressed += pointerPressedHandler;
                device.PointerMoved += pointerMovedHandler;
                device.PointerReleased += pointerReleasedHandler;
            }
        }

        protected override void OnDisable()
        {
            // Unsubscribe from DeviceProxy events
            var device = DeviceProxy.Instance;
            if (device != null)
            {
                device.PointerPressed -= pointerPressedHandler;
                device.PointerMoved -= pointerMovedHandler;
                device.PointerReleased -= pointerReleasedHandler;
            }

            base.OnDisable();
        }

        // Resets pointer before putting it in the pool
        private void resetPointer(Pointer p)
        {
            p.INTERNAL_Reset();
        }

        // Adds a pointer into the system
        private TouchPointer internalAddTouch(Vector2 position)
        {
            // Get a pointer from the pool
            var pointer = touchPool.Get();
            // Set its (remapped) position
            pointer.Position = remapCoordinates(position);
            // Set its buttons as "button one pressed this frame"
            pointer.Buttons |= Pointer.PointerButtonState.FirstButtonDown | Pointer.PointerButtonState.FirstButtonPressed;
            // Add the pointer to the system
            addPointer(pointer);
            // Press the pointer
            pressPointer(pointer);
            return pointer;
        }

        // Returns a copy of a cancelled pointer
        private TouchPointer internalReturnTouch(TouchPointer pointer)
        {
            // Get a pointer from the pool
            var newPointer = touchPool.Get();
            // Copy data from the old pointer
            newPointer.CopyFrom(pointer);
            // Set its buttons as "button one pressed this frame"
            pointer.Buttons |= Pointer.PointerButtonState.FirstButtonDown | Pointer.PointerButtonState.FirstButtonPressed;
            // Set flags
            newPointer.Flags |= Pointer.FLAG_RETURNED;
            // Add the pointer to the system
            addPointer(newPointer);
            // Press the pointer
            pressPointer(newPointer);
            return newPointer;
        }

        // Removes a pointer from the system
        private TouchPointer internalRemoveTouch(int id)
        {
            TouchPointer pointer;
            // Check if we have a pointer with such id
            if (!deviceIdToTouch.TryGetValue(id, out pointer)) return null;

            releasePointer(pointer);
            removePointer(pointer);
            return pointer;
        }

        private void pointerPressedHandler(object sender, DeviceProxyEventArgs e) 
        {
            Debug.LogFormat("Pointer {0} added at {1}.", e.Id, e.Position);
            lock (this)
            {
                deviceIdToTouch.Add(e.Id, internalAddTouch(new Vector2(e.Position.x / Width * screenWidth, e.Position.y / Height * screenHeight)));
            }
        }

        private void pointerMovedHandler(object sender, DeviceProxyEventArgs e) 
        {
            Debug.LogFormat("Pointer {0} moved to {1}.", e.Id, e.Position);
            lock (this)
            {
                TouchPointer touch;
                if (!deviceIdToTouch.TryGetValue(e.Id, out touch)) return;

                // Update to new position
                touch.Position = remapCoordinates(new Vector2(e.Position.x / Width * screenWidth, e.Position.y / Height * screenHeight));
                updatePointer(touch);
            }
        }

        private void pointerReleasedHandler(object sender, DeviceProxyEventArgs e) 
        {
            Debug.LogFormat("Pointer {0} removed.", e.Id);
            lock (this)
            {
                internalRemoveTouch(e.Id);
                deviceIdToTouch.Remove(e.Id);
            }
        }
    }
}

And DeviceProxy source code for testing.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Timers;
using System;

namespace MyDevice
{
    public class DeviceProxy 
    {

        const float UPDATE_INTERVAL = 10f;
        const float CREATE_INTERVAL = 2000f;
        const int MAX_POINTERS = 10;
        const float WIDTH_IN_M = 4f;
        const float HEIGHT_IN_M = 3f;
        const float MOVEMENT_SPEED = 0.3f; // m/sec

        public static DeviceProxy Instance
        {
            get 
            {
                if (instance == null)
                {
                    instance = new DeviceProxy();
                }
                return instance;
            }
        }

        public event EventHandler<DeviceProxyEventArgs> PointerPressed;
        public event EventHandler<DeviceProxyEventArgs> PointerMoved;
        public event EventHandler<DeviceProxyEventArgs> PointerReleased;

        private static DeviceProxy instance;

        private Timer updateTimer, createTimer;
        private Dictionary<int, Vector2> pointers = new Dictionary<int, Vector2>(MAX_POINTERS);
        private System.Random rnd;
        private float speed = MOVEMENT_SPEED * UPDATE_INTERVAL / 1000f;
        private int pointerId = 0;

        public DeviceProxy()
        {
            rnd = new System.Random();
            createTimer = new Timer(CREATE_INTERVAL);
            createTimer.Start();
            createTimer.Elapsed += (sender, e) => 
            {
                addPointer();
            };

            updateTimer = new Timer(UPDATE_INTERVAL);
            updateTimer.Start();
            updateTimer.Elapsed += (sender, e) => 
            {
                var keys = new List<int>(pointers.Keys);
                foreach (var id in keys)
                {
                    var pos = pointers[id];
                    pos += new Vector2((float)(rnd.NextDouble()) - 0.5f, (float)(rnd.NextDouble()) - 0.5f).normalized * speed;
                    pointers[id] = pos;
                    if (PointerMoved != null) PointerMoved(this, new DeviceProxyEventArgs(){Id = id, Position = pos});
                }
            };
        }

        private void addPointer()
        {
            var toRemove = pointerId - MAX_POINTERS;
            if (toRemove >= 0)
            {
                pointers.Remove(toRemove);
                if (PointerReleased != null) PointerReleased(this, new DeviceProxyEventArgs(){Id = toRemove});
            }

            var pos = new Vector2((float)(WIDTH_IN_M * rnd.NextDouble()), (float)(HEIGHT_IN_M * rnd.NextDouble()));
            pointers.Add(pointerId, pos);
            if (PointerPressed != null) PointerPressed(this, new DeviceProxyEventArgs(){Id = pointerId, Position = pos});
            pointerId++;
        }
    }

    public class DeviceProxyEventArgs : EventArgs
    {
        public int Id;
        public Vector2 Position;
    }
}
You can’t perform that action at this time.