FluidSynth/FluidLite MIDI playback support (SF2/SF3 files)

[fdelapena]

Despite SDL_mixer already supports FluidSynth, some ports may take advantage of FluidLite because it is a lightweight version of FluidSynth not requiring GLib. However, it still requires some additional work to provide standard MIDI file (SMF) parsing (which might be reused and mapped somehow from FMMIDI loader and/or try to import some minimal stuff from the larger FluidSynth).

The advantage of SF2/SF3 is a single file, replaceable by users (at least a couple of users asked for this feature), and SF3 uses Vorbis for samples. I've compressed the 30MB (28MB in zip) GeneralUser.sf2 into a 6.5MB SF3 (3.7MB in zip). This allows a larger, more complete, higher quality sound bank (better drums, no missing instruments) for ports such Android, saving a significant amount of package space without sacrificing sound quality.

FluidSynth/FluidLite license is LGPLv2.1, which is GPLv3 compatible.

There is also FluidSynth Lite, which strips glib too, provides .mid support, but not SF3 support currently.

[carstene1ns]

Here is an incomplete branch that could be of some help:
master...carstene1ns:fluidsynth_decoder
Basically needs build system stuff for library detection and a bit of polish.

Open question is how to provide the soundfont.

[Ghabry]

Because we still have no global config support yet a environment variable is probably a solution. Which other software uses fluidsynth for taking a look?

For android it can be a GUI option. More complexity: When this is in maybe shipping timidity could be stopped for Android at least and provide a GUI option to either download the GUS stuff or a free soundfont. (Mkxp has one linked)

[fdelapena]

After doing several experiments with .sf3 files, there is a significant latency while loading midi files, so we should stick into .sf2 for performance reasons.

[fdelapena]

I'd suggest basic support for looking soundfont.sf2 in the Player working folder or in the game folder until #666 gets decided.

[Ghabry]

I used the work of c1 and the sequencer API of FmMidi to provide output via FluidLite. FluidLite is very easy to build but it needs an external midi sequencer which we have (and lacks DLS support). :)

https://github.com/Ghabry/easyrpg-player/tree/fluidsynth
(Is based on the Filesystem branch to reduce rebase pain)

Here the relevant commit: Ghabry@e830396

Open Todos:

• Detect if Fluidsynth or FluidLite is used in CMake/Autotools. Imo we should support both.
• Not all Midi features supported yet, but I hear something already :)

[Ghabry]

The Fluidsynth API has one issue: The Soundfont belongs to the synthesizer. This means everytime when a MIDI file is opened FluidSynth has to parse the entire soundfont. :/

An optimisation is keeping the synthethizer alive and just recycle it when a new File is played. Because SE are usually not MIDI - only the 1 BGM - this will work properly in most cases.

[Ghabry]

Is now ready imo: https://github.com/Ghabry/easyrpg-player/tree/fluidsynth

Not opening a PR yet, depends on #2245 and #2189 and throws compile errors right now because I have them applied locally.

Features:

• Refactored MIDI Api. Is now managed by AudioMidi which handles instantiation of an appropriate MidiDecoder. A GenericMidiDecoder implements most of the API already and manages a FmMidi Sequencer and an underlying MIDI library.
• When the MIDI library supports external sequencers only FillBuffer and Midi messages must be implemented. Ticks, loops etc. are managed by GenericMidiDecoder.
• The Api also handles libraries that lack external sequencer support (WildMidi). GenericMidiDecoder is aware of it and will call appropriate functions and do sample counting. For WildMidi this means that Seek to implement Loops will work!
• FluidSynth/Lite support (Synth has priority in detection)
• The Fluid synth instance is recycled for BGMs to avoid long delays when loading the soundfont
• VIO for soundfonts through Filesystem API

TBD: Good default filenames/search locations for Soundfonts

[Ghabry]

The env var SDL_SOUNDFONTS could be reused.

Gstreamer directory search strategy: https://github.com/GStreamer/gst-plugins-bad/blob/master/ext/fluidsynth/gstfluiddec.c#L496

Out of scope for now: Talk to ALSA directly and use the running (fluidsynth or whatever) midi deamon like aplay and Wine do.