Gestolumina

Gesture Interpreted Light, Sound and Haptics. Towards a Framework for Universal Music Design

Team Members

Shawn Trail, PhD	Nate Hergert	Myles De Bastion
CymaSpace	CymaSpace	CymaSpace
www.cymaspace.org	www.cymaspace.org	www.cymaspace.org
Portland, Oregon	Portland, Oregon	Portland, Oregon
shawntrail@gmail.com	nate@cymaspace.org	myles@cymaspace.org

Dillon Simeone	Doga Cavdir, PhD	Duncan MacConnell, PhD
CymaSpace	Medialogy	Avid
www.cymaspace.org	Aalborg University	linkedIn
Portland, Oregon	Copnenhagen, Denmark	Montreal, Quebec, Canada
dillonsimeone@gmail.com	Idoga.cadvir@gmail.com	duncanmacconnell@gmail.com

Intro

Two different styles of bracelets are being built as custom musical instruments, one with pixelblaze and another with ESP32. The purpose of those bracelets are to control nearly haptics and LED systems via sensors detecting the wearer's gestures and movements.

ESP32 Bracelet (GestoLumina)

A low cost but extremely powerful approach, very beginner-friendly due to being programmed in arduino. Require more elbow grease to get up and running similar to the pixelblaze. However, the bluetooth in nearly all ESP32s makes them very easy to program as bluetooth musical instruments!

Pixelblaze Bracelet

The tool of choice for visualizing sounds with leds. This was the first quick rough prototype, since it was easy to order the sensor extension which comes with all sensors required for the project. Custom firmwork, somewhat accessiable due to its programming language being similar to javascript. No support for TFT displays, arduino libraries, etc. Not as straightfoward to set up as a musical instrument.

Misc

Prezi Show

Confluence built this prezi show for workshops, to quickly give people a crash course into how CymaSpace handles sounds to lights and music theory.

View Prezi

Videos

Misc

• Lasering parts out
• Random reverse engineering for bracelet straps

Pixelblaze

• Making chassis
• Testing frequency reactive code
• Testing beats per minute mode

GestoLumina

• Testing Hallway Monitor
• Testing ring led reacting to sounds
• Testing MPU-6050 1
• Testing MPU-6050 2
• Testing MPU-6050 3
• Testing haptics
• Inching toward final form... Too bulky! The pins coming out of the ESP32 board are evil!
• Second Prototype... It's wearable! The ESP32 is soldered onto a protoboard, and wires to that, which managed to cut down on volume.

Pictures

Pixelblaze

Cutting up and soldering together LED strips so they'll fit on the living hinge bracelet.	The pixelblaze bracelet lives!	Making a new bracelet and wiring the leds differently. Previously, they was wired in parallel.

ESP32 (GestoLumina)

The moment when it was realized that most of the sensors would fit neatly into wooden rings made by a laser cutter.	Testing which ring size seems to fit on people's fingertips the best. 18mm~ turned out to be the ideal size for those ring designs.	Assembling sensors on the rings, it went by pretty nicely.
It fits!	Displaying all sensor readouts on the screen for easier development and troubleshooting.	We was careful to laser the box in a way where sensors and the like can be added or removed without desoldering wires, breaking the box, etc.
Testing the sparkfun haptic driver, it turned out to be really weak. Barely noticeable at max power. That's 3v haptics for you!	Added a L298N H-bridge to control the haptics with, much better!	This half glove is what the entire system will be sewn to eventually!
Protoboard used! Much better. It was at this point where the team realized that a custom PCB would be much better.	After some hot glue and stuff was added... Bam bam, it's wearable now! And can send midi notes with gestures!	Designing the custom PCB for the first time... This is just the beginning! A lot of things to study, textbooks to read and videos to watch.