Tracktion for Sound2Synth

This plug-in is built based on the tracktion_enngine and is specialized for using the Sound2Synth Dexed plug-in. Please note that the included tracktion_engine module is modified, which is different from the original github repository.

This plug-in is designed for replacing PyAU, which functionality is to convert or batch convert .json formatted Dexed parameters into rendered audio.

A video tutorial is available at this link.

Setup

An off-the-shelf version of this plug-in is available under the Builds/ folder. For example, on Mac, use Builds/MacOSX/build/Release/PluginDemo.app.

Please make sure the Sound2Synth Dexed is correctly installed before using this plug-in. Currently only AU version of Sound2Synth Dexed is supported.

1. Plug-in Setup: A scan over plug-ins is required when running the plug-in for the first time. Please select Plugins - Options - Scan for new or updated AudioUnit plug-ins and wait for the process to finish. Please check that Dexed in AudioUnit format is detected.

2. Dexed Setup: On the right of Track 1, select + - Digital Suburban - Dexed to load the Dexed plug-in. By pressing D the Dexed interface will pop up.

3. MIDI Setup: When running Sound2Synth, default midi files (each contain one and only one note played) are generated under midi_settings. For convenience, we provided the default generated midi file here at MIDI/60.mid. In the plug-in first select Track 1 to make sure it is highlighted, then select Import Midi and choose the provided midi file (or any other midi file you prefer).

Usage

Use Render to directly render using the current configuration of the loaded Dexed plug-in, or select Batch and choose the folder of the .json formatted parameters to batch render.

Citation

@inproceedings{ijcai2022-682,
  title     = {Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation},
  author    = {Chen, Zui and Jing, Yansen and Yuan, Shengcheng and Xu, Yifei and Wu, Jian and Zhao, Hang},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {4921--4928},
  year      = {2022},
  month     = {7},
  note      = {AI and Arts},
  doi       = {10.24963/ijcai.2022/682},
  url       = {https://doi.org/10.24963/ijcai.2022/682},
}