This is the result of the work for my Master Thesis in Computer Engineering at "Università di Pavia", Italy. Eyerpheus is a collection of design experiments about gaze controlled Virtual Digital Music Instruments (DMIs).
Among them, the main one is Netytar, a DMI completely designed for gaze interaction. Its interface consists of an array of colored dots connected by lines, and can be described as isomorphic (the geometrical shape of a sequence is transposition-independent). A complete description can be found in the paper Davanzo, N., Dondi, P., Mosconi, M., & Porta, M. (2018). Playing Music with the Eyes Through an Isomorphic Interface presented at ETRA COGAIN '18, Warsaw, Poland, 14-17 Jun 2018. Please cite this article if you use this prject for your research.
- Windows operating system (tested on Windows 10. Not available for Linux :( still not supported by Tobii eyetrackers)
- Tobii EyeX eyetracker (could work also with Tobii 4C. Never tested)
- (Optional but recommended) A VST host with some good VST effects (included in the instructions)
- (Optional) XboX 360 Gamepad Controller (for FFB effects)
- Clone this repository on your HD. You'll need every single file;
- We'll need to have a MIDI loop port on our PC. Download loopMIDI and install it. Create a loop port with an arbitrary name. Make sure to have loopMIDI running when you use Eyerpheus;
- For the VSTs, I personally recommend VSTHost. Download and install it. It's quite simple to use: you just have to drag and drop the VST's DLL files inside, then connect them to the output. Make sure that the desired VST takes the loopMIDI port as input: to do so, click on the left MIDI connector image on the VST, then select the correct port as input. Again, make sure VSTHost is running while using Eyerpheus;
- Unzip and install all the fonts contained in the "digital-7.zip" file;
- Launch Eyerpheus using this file: WPFeyerpheus/WPFeyerpheus/bin/debug/WPFeyerpheus.exe.
- Gaze: selects a note. A white flash will appear to highlight the selection;
- S key: plays the selected note (use this if no other controller is available);
- Autoscroll button: triggers the automatic scrolling system (see the Interface guide);
- If an Xbox 360 controller is connected, it will vibrate at every note change.
For a greater insight into Netytar, please refer to the paper [Playing Music With the Eyes Through an Isomorphic Interface](Playing music with the eyes through an isomorphic interface.pdf) "Playing music with the eyes through an isomorphic interface"), available in this repository.
- Netytar / WickiEyeden selector: you can choose which instrument you want to play here. Please note that WickiEyeden is incomplete and under testing.
Scale selector: this selector is used to highlight a particular scale in Netytar's "web", an useful playing aid. A major scale will be colored in red, a minor in blue. Also the notes of the respective scale will be highlighted;
Note informations: the first mini-screen indicates the current note pitch (in MIDI note), the second indicates the note's name, while the third indicates if the note is currently being played or the instrument is silent (B stands for "Blowing". Netytar is thought to be used with an external sip-and-puff input device);
Visual metronome tempo regulation: this is used to regulate the tempo of the visual metronome, an experimental tool that flashes light into the interface at the selected tempo.
Triggers: the first one triggers the "autoscroll" functionality (necessary to play both instruments): the instrument's interface will follow the gaze automatically. The second one triggers the visual metronome. The third one tests the FFB effect: if an Xbox 360 controller is connected to the usb port, it will vibrate.
MIDI channel selector: this is used to select the MIDI output channel. If you don't hear any sound, try a different channel.
