Symphonary is a program designed to teach people how to play a musical instrument. While many people would probably like to play an instrument (or have failed at some point in their lives), it can be quite daunting for a beginner to learn to play something on their own. Although you can easily purchase books containing sheet music, beginner sheet music books don't exactly contain the kind of music that most people would want to listen to, and often don't include any way to hear what the music sounds like, aside from playing it yourself and hoping you're correct. Besides, learning to read sheet music can be difficult for people just starting out, and getting music that isn't for piano can be difficult. Symphonary attempts to address some of these problems, by:
-- Not showing sheet music. The game is played in the style of Guitar Hero / DDR, where notes fall down the screen. No knowledge of sheet music is needed.
-- Music is automatically converted to / from any instrument, wherever possible. For example, all flute music can be played on a piano, though not all piano music can be played on a flute (as a piano can play any number of notes simultaneously). Music is transposed on-the-fly if it isn't compatible with what you're playing with. Note that this doesn't mean that you have the skill to play a given song, or that a given song will still sound nice when played on a certain instrument. However, the software shouldn't stop you from trying.
-- You can play any MIDI song you can find, not just sheets belonging to your instrument. This includes popular songs, which may be more desirable to some people.
-- Scoring the user as they play. At present, nothing happens with this data, although charts could be generated showing a user's progress over time, or letting them automatically select the parts of a song they perform poorly. Gamification and proper feedback may be the difference between someone succeeding or failing.
-- Songs with accompaniment allow the user to hear the other parts of the song, and selectively enable parts. They can hear what it sounds like without them, hear what it sounds like with them, and hear what it would sound like with only themselves performing.
The software works by connecting one's computer to a physical musical instrument, and then sending information back to the computer about the state of the instrument in real time. For a guitar, this data includes the strings that are presently being pushed down (and where they're being pressed) and the intensity that each string is vibrating at. For a trumpet, this would include data on the air pressure going into the mouthpiece and the state of each valve. The software has some support for flute, violin, guitar and bass, although other instruments can be added fairly easily, as most of the code is generic.
Unfortunately, it is not so easy to connect an instrument to one's computer. While you can easily buy a MIDI keyboard for less than $100 (and use the excellent program Synthesia [http://www.synthesiagame.com/], which does roughly the same thing solely for piano), MIDI guitars (and other instruments that aren't pianos) either don't exist, or suffer from a number of flaws, including that:
i) They use microphones or other pickups, meaning there's substantial latency between playing a note and seeing it on screen.
ii) They're made of plastic, and don't have actual strings.
iii) They cost a lot, or are otherwise difficult to find, and would be off-putting to beginners that don't wish to invest thousands.
While iii can be bypassed with money, ii and i aren't quite so easy to work around. It's easy to get discouraged if you're learning on a plastic guitar, when you discover that your skills don't exactly translate to a real guitar. Likewise, if you're using a pickup, the amount of latency limits the usefulness of the program. Below are some examples of things that are only possible with an instrument capable of communicating in real time:
-- By reading data about where your fingers are, we can predict whether or not you're capable of playing the right notes in time. If you can't make it, we can pause the game automatically until your fingers move into the correct position.
-- You could practice strumming and fingering (or blowing, etc) separately, to learn one hand at a time. The computer would know how well each part was being played, and would be capable of performing the roles you aren't.
-- In the case of an electric guitar (or drums, etc), the instrument could be silenced by the computer, for better or worse. A player who might not want to disturb others could listen to the synthesized equivalent of themselves playing, via headphones, instead of their actual instrument's noises.
-- Recordings of everything played could be generated by the instrument itself. You could generate sheet music or tabs after having played a song once, or play back a song that was performed earlier.
-- Graphs can be generated showing where in each song you screwed up, with exact details on why you didn't play it according to the way it was written (too much air pressure, slow fingers, etc).
As no existing hardware met our needs at the time of development, Symphonary uses a custom protocol to pass data back to the computer, depending on the instrument. If you'd like to use a working physical instrument, you'll need to physically build a compatible piece of hardware, which may be daunting depending on your level of experience with electronics, or add support for something you might have to the project. However, if you just want to test the software, you can do so without anything hooked up (it will still work, just without any kind of on-screen status about your instrument).
Alternatively, you can write the sample_guitar program to an Arduino and connect that to your computer, which will impersonate a guitar. The protocol just consists of a string of text sent over serial, which contains data about which strings are currently being held down (AAAAAA is all open frets, BAAAAA has the first string held at fret #1 while all others are open, etc).
Note that the software doesn't support MIDI-in, as MIDI doesn't carry the type of data we need in most cases. While it works perfectly with a piano (where one key represents one note), MIDI won't work nicely with a guitar. While MIDI will tell you what note was played, it won't tell you which string it was played on (without non-standard hacks), nor will it tell you if a string is being pressed when it isn't making a sound (without non-standard hacks). However, if you wanted, you could add it in fairly quickly, as one of the libraries Symphonary uses has support for reading MIDI input.
Symphonary is easy to build - just open it in Visual Studio 2010 or higher and press compile. It's written in C# using WPF.
This is what it looks like after a MIDI has been loaded. COM3 represents the guitar plugged in to the serial port, although exact numbers will vary based on machine details. The specific part of the song that you want to play is selected (in this case, the electric guitar). Note that you don't have to select a guitar track just because you happen to have a guitar, though the guitar track in this case contains the most straightforward part.
This is what playback looks like. The area on the left shows the current status of your instrument. If no instrument is plugged in, no numbers will be displayed on the left-hand side. In this case, we have a guitar with all open strings, except the lowest note is held at the fifth fret. If the player wanted to score points here, they'd move their fingers on the second-lowest string to the fourth fret pretty quickly, before the notes pass under the left side.
I've since switched to OS X, so here's what full screen looks like running in Windows 8.1 on VMware Fusion. The software was originally developed on Windows 7 though, but appears to be fully compatible. We had primitive support for flute, guitar, and bass in an earlier build, though after the GUI was rewritten, effort was only spent repairing guitar support. However, fixing support for existing instruments is trivial, as is adding a new instrument.
Here's another example of a song, at a much higher level of difficulty. Note the score in the upper right hand corner, which shows that the person playing right now probably has no business ever performing publicly.
Copyright (C) 2011-2012 Xingzhe (Travis) Lu and Tyler Freedman.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License.