Augmenting the touch interaction on smartphones and tablets with information from a wrist worn IMU.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
Evaluation Data

Expressy Video

Expressy: Using a Wrist-worn IMU to Add Expressiveness to Touch-based Interactions

This repository relates to a CHI 16' publication "Expressy: Using a Wrist-worn IMU to Add Expressiveness to Touch-based Interactions".


Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions that rivals that experienced by the painter and those completing other skilled physical tasks. We investigate how data about hand movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We begin by introducing a conceptual model that formalizes a design space of possible expressive touch interactions. We then describe and evaluate Expressy, an approach that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We conclude by describing a number of sample applications, which demonstrate the enhanced, expressive interaction capabilities made possible by Expressy.


Running the Application

  1. Download Big Buck Bunny video from and place in Expressy\bbb_sunflower.mp4.
  2. Now you can open the Expressy.xcworkspace workspace and deploy it to your device.