Sample Generation / Audio Export API

[Danielku15]

Motivation

Most details have been discussed here: #1935

The plan of alphaTab is to support eventually the integration with external audio and video sources in an easy-to-use fashion (e.g. by linking to an audio or video element, or other player frameworks like youtube).

In some scenarios a hybrid audio is desired where an external and synthesized audio play together.

By having APIs through which devs can synthesize the whole song audio and obtain the samples an external component can handle the synchronization and playback. Together with the External Audio Cursor API alphaTab will then show the correct information.

Proposed Feature.

alphaTab should provide separate APIs to generate the raw audio samples for a song for custom use. Either for the use in direct playback or simply for saving the audio to disk.

As raw PCM audio can grow very large a streaming style API is required. Via the API object a new audio generator can be created. This audio generator is separately initialized with settings like:

• Sample Rate
• SoundFonts to use (if the player is active would be good to reuse the existing worker and synth state with loaded fonts and midi).
• Format (Raw PCM Shorts, Raw PCM Float, OGG Vorbis compressed).

The new API would heavily use promises to synchronize with the background worker doing the work. On every render call a buffer size can be provided how many samples should be synthesized (when using OGG Vorbis compression we have to check the exact framing logic).

OGG Vorbis

alphaTab only has an OGG Vorbis decoder (lately since SF3 support). Unfortunately NVorbis has no encoder we could port over. But likely a lot of the algorithms can be reused. There are many libs in the market we can peek into learning the bits without digesting the raw specs.

Configuring the audio generator.

The audio generator provides a similar API like the main player allowing control over the track volumes, playback speed, metronome etc.

Metronome or Instrument isolation

If somebody wants an isolated metronome track or one audio stream per track, multiple audio generators have to be created with different configurations.

Implementation Details

• Synthesizing should be done via alphaSynth worker (using new commands).
• The existing main playback worker should be able to handle the generation.
• Every created audio generator has an own ID which talks to a dedicated internal synth instance.
• We use promises with request-reply IDs to handle the communication and call synchronization.
• If the main playback worker is reused, it should auto-init with the soundfont and midi already loaded.
• If no player is enabled a new dedicated worker is spawned and freshly initialized with the midi and soundfont.
• The audio generators need to be disposed to free internally the synth state and workers.