Analog sensors (balance boards, accelerometers, light sensors, distance sensors) can be used together with music or video material. During an experiment which monitors a music performance it might be a requirement to record music, video and analog sensor in synchronously. The software here allows this in an efficient and practical way with standard DAW software.
In order to do this analog sensors are attached to a microcontroller which can present itself as class compliant MIDI devices. Low latency recording of analog sensor data, then, boils down to recording MIDI events from a standard MIDI instrument. The system consists of a programmed microcontroller, a standardized MIDI protocol and a script to convert the recorded MIDI track to a text file with timestamped measurement values which can be analyzed.
The name of the system hints at the MIDI protocol being morphed into something it was not designed for. From a system designed for communication between instruments into a horrible vermin, a bug, a dung beetle?
Use of the MIDImorphosis
First a microconroller with up to eight analog sensors needs to be prepared. See below for example code for a Teensy microcontroller. Crucially, this device needs to be programmed as “MIDI Device”.
Next the sensor data needs to be recorded. This can be done in Ardour, Ableton or any other DAW software system. After recording the midi data it needs to be exported.
Once a midi file is recorded, it needs to be converted to a format that is more practical to analyze: a csv file with timestamps in seconds and values. A Ruby script is available that does this conversion. To convert a midi file to a csv file with timestamped midi events in seconds you need Ruby and midicsv on your system. See installation if this is not already installed.
To execute the script it needs at least two arguments. The first argument is the BPM value that was used to record the MIDI file. Since some DAW’s (Ableton) do not encode this in the MIDI file it needs to be provided. The second and following arguments are MIDI files. Each midi file is converted and a csv file is saved in the same folder as the midi file with a csv extension.
ruby the_midimorphosis.rb 120 [midi_input_01.midi other_midi_file.midi..]
ruby the_midimorphosis.rb 120 folder/*.midi
The result is a csv file that uses the same time-line as the audio recordings (and potentially other recording modalities).
The binary MIDI file is decoded with the midicsv tool and this decoded information is filtered with Ruby. The midicsv program needs to be accessible on the path of your machine. To do this the following can be done:
On Mac OS X, first install homebrew, and then execute
brew install midicsv
On debian like systems
sudo apt-get install midicsv
For windows systems, an executable should be available together with this software. Alternatively, download the midicsv executables.
A ruby system should be installed on your system as well. On macOS this is done automatically. On Debian use
sudo apt-get install ruby and on Windows the Ruby installer.
Testing and Visualization
To get started there is also test-data provided in the
test-data folder. To convert that test data, use
ruby scripts/the_midimorphosis.rb 120 test-data/balance_board.mid
Each sensor encodes measurements in MIDI CC messages. Each CC message has a CC number with a predefined purpose. Next to this number it also has a value between 0-127 (7bits). More information on MIDI messages can be found online. MIDI CC messages have the advantage with respect to NOTE ON messages that they do not need a NOTE off message and are for the most part ignored by software synthesizers.
Analog measurements on a Teensy are encoded in 13 bits. By combining two CC messages with 7 bit values we have 14 bits at our disposal. Furthermore, to prevent that the CC messages have some effect (e.g. changing volume) we use the undefined CC numbers between 102 - 119. Each analog measurement is also encoded on a separate MIDI channel. There are 16 MIDI channels available. This means that analog measurements can be recorded from a single analog input (e.g. A0) if you select only messages on a single MIDI channel (e.g. MIDI channel 1). If all channels are allowed the analog input can be determined by the MIDI CC number
So each analog measurement is multiplied by two (13 to 14 bits), encoded into two MIDI CC messages which are sent together. First the most significant bits (MSB) are sent on even MIDI CC numbers (starting from 0×66) immediately followed by the least significant bits (LSB) on odd MIDI CC numbers (starting from 0×67). The following table summarizes this:
|Teensy Input||MIDI Message type (Hex)||MIDI Channel||MIDI CC Number||MIDI CC Number interpretation|
|A0||0XB0 176||1||0×66 102||MSB|
|A1||0XB1 177||2||0×68 104||MSB|
|A7||0XB8 184||9||0×74 116||LSB|
Additionally an extra MIDI CC message is send on controller number 88 on channel 15. This message is sent at the start of each measurement and can be used to split messages and ensure an even sampling even when only changing midi messages are stored (as is common).
There is also an option to send a NOTE_ON and NOTE_OFF event pair every second in order to verify that the system is sending data. Make sure that these events are ignored by any software synth.
MIDI Timing precision
Midi encodes time in ticks. The tempo is specified as the Number of microseconds per quarter note, between 1 and 16777215. A value of 500000 corresponds to 120 quarter notes (“beats”) per minute. To convert beats per minute to a Tempo value, take the quotient from dividing 60 000 000 by the beats per minute. To calculate only the size of this tick the midi_csv program can be called as follows:
$ ruby scripts/the_midimorphosis.rb 150 -p test-data/balance_board.mid Processing /Users/joren/Dropbox/UGent/Builds/2017.miditocsv/github/test-data/balance_board.mid ... Did not find tempo. Will use 150 BPM balance_board.mid 4.166667 milliseconds
For synchronization purposes it is important that it is known when measurements are taken. The most easy way to synchronize is to use an external TTL clock. This external clock can then direct the device to take a measurement e.g. 120 times each second.
By default the microcontroller waits for a clock input. There is, however, also an option to generate a stable clock internally using interval timers. To enable these either configure the Teensy to start the internal clock automatically or send a PITCH BEND midi message to the device with the sample rate in Hz divided by 127 as the first byte (MSB) and rest as the LSB.
Micro-controller configuration – Teensy code
To create a micro-controller compatible with this setup, it needs to be programmed as a class compliant MIDI device. It also needs to follow the MIDI protocol described above. Synchronization can be done with either an internal clocking mechanism or by following an external clock.
The analog measurements are sent as 14 bit words. The Teensy supports 13 bit measurements so these are multiplied by two. Other precisions should be converted to 14 bits as well.
For reference, please consult the Teensy code