Fusion is an open source, modular, cross-platform interaction framework designed to rapidly prototype and develop multi-modal application interfaces. It seeks to expose an easy-to-consume API for user input in multiple simultaneous input modes. Initially, we provide modules for audio input to be recognized by a grammar and Sphinx ASR, and touch/mouse input. See Put That There 1 and Put That There 2 for an example of multimodal interaction.
See the Fusion page at the NUI Group's website for a video of Fusion's demo in action.
This project was initially written for the Natural User Interface Group as part of Google Summer of Code 2011.
Cross-Platform Fusion is fully cross-platform. It runs on Windows, Linux, MacOSX and FreeBSD. The same configurations and models apply across platforms, so if your application is cross-platform, we support it.
Daemon Fusion runs as a daemon process. That means you can launch it on a different headless computer if you want (although currently some things depend on the daemon running on localhost, and responsiveness could be an issue).
Threading With Fusion, you have the ability to enable threading for any module you want. That means you are finally able to use your Core 2 Duo / Quad Core to its full potential. Obviously, you can still use non-threaded mode.
Remote API Fusion has a JSON API to control all its settings and adjust your pipeline remotely and on the fly. Interaction events are also served as JSON objects.
HTML5 Administration Fusion's web interface makes use of the most recent developments in web programming with open standards like HTML5. All that is required is a compatible HTML5 web browser. As of today, Firefox 3.6, Safari and Chromium have successfully run the web interface.
Fusion currently runs on Windows 7/8, OS X 10.6+, and Linux (tested on Ubuntu 11.04+). It currently uses the SCons tool to build on all platforms.
This means you have to install SCons to be able to compile Fusion. The platform specific compile notes below include instruction on how to install SCons. Once you have SCons installed open a command line in the Fusion directory (the one with this README in it) and run the following command:
To build fusion (and dependencies if needed) run: scons
To clean fusion (remove any intermediate files, output libs and executables): scons -c (or 'scons --clean')
To also clean the dependencies built in contrib: scons -c --clean-contrib
For more info about SCons, other options, or how to install it on any system, you can check the SCons user guide at: http://www.scons.org/doc/production/HTML/scons-user.html
To build Fusion on Windows, you need Visual C++ 2010 and SCons (which also requires Python).
Get Visual C++ Express 2010 (not needed if you have Visual Studio 2010). Download the free version here: http://www.microsoft.com/express/Downloads/#2010-Visual-CPP
OK, you're ready to build. If you installed everything to default locations just run:
If scons command is not found, try
Now you should be able to start fusion. Just double click the fusion executable or run it from the command line to use a config:
fusion -l config/ballworld/ballworld.cfg
OS X and Linux
On OS X and Linux, Fusion is also built with SCons (http://www.scons.org).
- Download SCons
- Unzip the file, open a terminal, and navigate to that directory:
python setup.py install
Now you're ready to build fusion. Change directory to the Fusion sources.
- First, you need to build rtaudio. This can take some configuration, so check out
contrib/rtaudio/install. Most times, though, you should just be able to build with
cd contrib/rtaudio && ./configure && make.
- Next, go back to the root of the project and
scons. This will (hopefully) build everything. Note: If you need to build fusion as 32bit, you can do so by forcing the -m32 flag on the compilers for both moved and all libs in contrib by running:
CC='gcc -m32' CXX='g++ -m32' scons
./ccf, and navigate to http://127.0.0.1:7500/ with a Canvas/HTML5/JS-enabled browser for configuration. Exit the server with
- Leave the config as-is and run the platform-specific Python file in
demofor a quick demo of Fusion's functionality.
OS X or Linux:
- Make sure you have Doxygen installed
- cd docs
- Run Doxygen on the Doxyfile in docs/