Collaborative Sound Design Based on Perceptual Mapping
Scala GLSL Java Shell
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
VPDSynth
data
libraries
src/main
.gitignore
.gitmodules
README.md
Settings.txt
build.sbt
g2.png
gtm_result_preset_clustered.csv
gtm_result_withfreqs_62k_presets_lat22500_rbf100_beta0.200000.csv_extracted.csv
gtm_result_withfreqs_FILTERED80.000000_62k_presets_lat4900_rbf100_beta0.200000.csv_extracted.csv
overview.png
sbt
sbt-launch.jar

README.md

SoundDesigner - Collaborative Sound Design Based on Perceptual Mapping

Screen-shot of prototype 1 User study

This is the original source code for the prototypical sound design application in the publication:

Klügel, Niklas and Becker, Timo and Groh Georg. "Designing Sound Collaboratively-Perceptually Motivated Audio Synthesis"

Abstract: In this contribution, we will discuss a prototype that allows a group of users to design sound collaboratively in real time using a multi-touch tabletop. We make use of a machine learning method to generate a mapping from perceptual audio features to synthesis parameters. This mapping is then used for visualization and interaction. Finally, we discuss the results of a comparative evaluation study.

which can be found here: http://www.nime.org/proceedings/2014/nime2014_339.pdf and http://arxiv.org/pdf/1406.6012 (extended version)

This application relies on a data set which has been published here as well, however the original sampling and timbre space construction method has not been released so far. It is originally a tabletop application but the framework also allows use with mouse input apart from TUIO and native win7 touch.

Dependencies:

  • reakt: https://github.com/lodsb/reakt
  • UltraCom: https://github.com/lodsb/UltraCom
  • the Vector phase distortion synth has to be installed (part of the repository)

  • checkout and build + sbt publish for each of these

  • The real-time audio synthesis uses SuperCollider / ScalaCollider, therefore an additional SuperCollider installation is necessary.

  • External midi input to drive the synthesis algorithms
  • Audio output device with multiple channels (for individial headphone outputs)