Skip to content
The Example-based Sensor Predictions (ESP) system applies machine learning to real-time sensor data.
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
Arduino Merge branch 'master' of Aug 11, 2016
Processing/BallDrop Add simple Processing example. Aug 2, 2016
Xcode/ESP reorder header search path in Xcode project Aug 11, 2017
cmake fix(build): test on RPi Oct 8, 2016
third-party Update ofxGrt. Oct 20, 2016
.gitmodules Update all git access to https access Aug 13, 2016
CMakeLists.txt Look for in third-party/grt/tmp. Oct 19, 2016
LICENSE.txt Update instruction to clone submodules Aug 5, 2017 Update openframeworks URL to https Jun 5, 2018 Use relative path Jul 27, 2016

ESP (Example-based Sensor Predictions)

Build Status

This project aims to help novices make sophisticated use of sensors in interactive projects through the application of machine learning. It works on Mac OS X, Windows, and Linux.

Check out our paper and video for detailed descriptions.

Installation for users

The easiest way to use ESP is through the Processing Development Environment (PDE). See the Processing ESP mode for details.

Installation for Developers

Pre-requisites: you'll need git plus Xcode on Mac OS X, Visual Studio on Windows, and CMake on Linux. To install, first clone this repository, then run the setup script:

git clone --recursive
cd ESP

This will clone the relevant git submodules and create some symbolic links.


The main application is an openFrameworks-based GUI application named, unsurprisingly, ESP. Below are instructions to run on different platform (we support Mac OS X, Windows, and Linux).

Choose the ESP example you want to run by uncommenting the corresponding line at user.cpp. See below for a list of available examples.

Arduino Project Hub has a more comprehensive tutorial on how to use the software.


Use Xcode to open the project at Xcode/ESP/ESP.xcodeproj. Select either the "ESP Debug" or "ESP Release" scheme (not "openFrameworks").

We also support using CMake on OS X to compile the project:

# Compile openFramework by compiling an emptyExample
xcodebuild -configuration Release -target emptyExample \
  -project "third-party/openFrameworks/scripts/templates/osx/emptyExample.xcodeproj"

# Build ESP
mkdir build
cd build
cmake ..
make -j8


We use CMake on Linux to compile the project. The compilation is a bit more involved, but should be easy to follow:

# Install required package
sudo apt-get -y install doxygen
sudo apt-get -y install cmake
sudo apt-get -y install libblas-dev

# Then build openFrameworks
sudo third-party/openFrameworks/scripts/ci/linux/
sudo third-party/openFrameworks/scripts/ci/linux/

# Build and install GRT
cd third-party/grt/build
mkdir -p tmp && cd tmp
sudo make install
cd ../../../../

# Build ESP
mkdir build
cd build
cmake ..
make -j4


See this setup guide


Many of these examples expect an Arduino board to be connected to the computer and running an appropriate sketch. Some of the sketches are hosted in this repository as well (see Arduino folder). Some examples are:

  • user_audio_beat.cpp: recognizes periodic sounds (e.g. dialtones, bells ringing, whistling) using an FFT and support vector machines algorithm. Works with your computer's built-in microphone.

  • user_color_sensor.cpp: detects objects by color using a naive Bayes classifier. Works with either the Adafruit TCS34725 breakout (using the sketch in Arduino/ColorSensor) or the SparkFun ISL29125 breakout (using the sketch in Arduino/ColorSensor_SparkFun_ISL29125). See documentation for the sensors for hookup information.

  • user_accelerometer_gesture.cpp: recognizes gestures using a dynamic time warping algorith. Works with either an ADXL335 accelerometer (using the Arduino/ADXL335 sketch) or the built-in accelerometer on an Arduino 101 (using the Arduino/Arduino101_Accelerometer sketch).

  • user_accelerometer_poses.cpp: recognizes the orientations of an object using a naive Bayes classifier. Works with accelerometers as for the user_accelerometer_gesture.cpp example.


See the online documentation of the ESP API.


These should be automatically installed by the setup script:


See LICENSE.txt for licensing information.

You can’t perform that action at this time.