Implement FluidSynth integration

[dreamer]

The patch integrating FluidSynth was developed a long time ago (first revision 9 years ago, last revision 3 years ago - it really waits that long for inclusion upstream?). It is currently being distributed via ECE.

In the past, I was very hesitant to integrate FluidSynth patch for a number of reasons:

1. I wanted to clean up first, before starting new work in the area. Without going into details… there are serious issues with DOSBox MIDI support - the demand for integrating Fluid might be caused by those issues.
2. I don't want to target Fluid 1.x API. It's old - Fluid is being actively developed and they have 2.x API released already.
3. When developing Boxtron I tested FluidSynth running as application thoroughly… and the results were very mixed - to the point, that I recommend Boxtron users to use Timidity++ by default. All that testing was around version 1.1 though (which is old and hopefully not representative of quality in 2.x).
4. After a brief look at patch itself, I have some reservations: MIDI synth should not run in the same process as emulation loop, properties do not use the same naming style as other options, INSTALL documentation is missing, user documentation is missing, build was never tested under Visual Studio, (and more) - all those issues can be easily fixed; patch itself is not that big nor invasive, it just needs some work.
5. libfluid 2.x was not available in repositories, which would mean integrating patch required bundling fuid in the repo, which means transition to cmake, yada, yada - all the same reasons as for Implement MT-32 emulation #257

What changed?

Over last 1-2 months (?), point (5) stopped being a problem, fluidsynth 2.x is now available in most repositories, including brew and vcpkg. That takes out reason (5), and it's huge.

Points (4) and (2) are just a matter of putting in some work - not a big deal. I still want to address (1), but after looking in details at the patch - this work can largely happen in parallel. (3) can be assessed only during testing.

Of course, integration of Fluid needs to be optional - the lib is not old nor tested enough to be propagated to all repositories, so some users will want to build without it. But the patch already kind-of handles it.

When?

Testing effort required would push 0.75.0 release too far away, so I won't merge any new MIDI-related changes before 0.75.0 release (here are tasks planned ATM for 0.75.0).

But if someone has free cycles and wants to push this work forward: the patch is imported on a side branch already - cherry-pick f786b9f and start working on fixes to problems I listed in (2) and (4) ;) If nobody will step up, I'll start work on this sometime after 0.75.0.

[kcgen]

Well laid-out plan @dreamer ! 📑

[bluddy]

Regarding part of point 4, I assume you mean using separate threads for midi synthesis. Fluidsynth already uses other threads internally, meaning that it doesn't burden the dosbox thread.

In general, the patch seems simple enough to me that it can just be merged. Further cleanup can proceed later on as a specific task. Most importantly, it'll allow more users to switch to dosbox-staging, who will then be invested in the project's success.

[dreamer]

Fluidsynth already uses other threads internally, meaning that it doesn't burden the dosbox thread.

We need proof to confirm this assertion. It's not hard, just generating flamegraphs and some testing - but that's part of development work needed to push the patch through the finish line.

Also, I am not going to merge patch in a state where CI does not verify the build using multiple compilers on multiple OSes. And that's the state the patch is in ATM :(.

[bluddy]

We need proof to confirm this assertion. It's not hard, just generating flamegraphs and some testing - but that's part of development work needed to push the patch through the finish line.

From the fluidsynth pages:

As soon as the audio driver is created, it will start playing. The audio driver creates a separate thread that uses the synthesizer object to generate the audio.

This is unlike munt, which has no notion of threading.

[kcgen]

Most importantly, it'll allow more users to switch to dosbox-staging

Great point @bluddy , I agree. Regarding the flamegraphs @dreamer mentioned, are you able to generate them?

I haven't had time to dig in yet; but if you can build and run fluidsynth (standalone) on a linux box, here are notes on how to generate them: https://github.com/brendangregg/FlameGraph/blob/master/README.md

[dreamer]

Without going into details, this is the way I generate flamegraphs (Linux only), before starting: the FlameGraph repo mentioned earlier needs to be cloned (for the scripts inside).

1. Build dosbox-staging with -g -O0 -fno-omit-frame-pointer
2. Start dosbox-staging, and note PID of the process
3. Start the benchmark / test inside dosbox, and immediately:
4. sudo perf record -F 99 -p <PID> -g -- sleep 120 -F 99 is probing stack at 99Hz - empirically, this seems like a good middle-ground, at least for me (we don't want too high value for probing, as it makes the results less realistic!); play the game or watch benchmark for 2 minutes
5. sudo perf script | ~/src/FlameGraph/stackcollapse-perf.pl > out.perf-folded
6. ~/src/FlameGraph/flamegraph.pl out.perf-folded > flamegraph-game-test-description.svg

Resulting SVG to be opened in the browser (it has js to allow for easier browsing and filtering the graph). I need to describe somewhere how to interpret the graph, but overall: there will definitely be a huge plateau of unrecognized stacks (that's dynrec-generated CPU emulation, we're not interested in that) and a tower, that can be clearly recognized as main dosbox "Normal" loop; narrow towers on top of normal loop are ok - plateaus on top of normal loop are bottlenecks. (The thing I'm not sure about is how to be 100% sure flamegraph does not include child threads - but for CD-DA this doesn't seem to be a problem).

If flamegraphs with FluidSynth integrated will look similar to graphs when game is playing music via CD-DA emulation - that's good. if fluidsynth will show up as plateaus covering ~5% of runtime stacks or more - that's bad and we'll need to investigate further.

edit I guess it would be helpful to list the games, that offer music playback both via CD-DA and MIDI to start preparing test cases. I think some candidates might be: HoMM2, Settlers 2, and System Shock.

[kcgen]

Instructions are great! Here's a zoom out of what I got (Jones in the Fast Lane, ~2500 cycles, FLAC CD-DA sequences back-to-back)

[dreamer]

Wow, this looks completely different than what I get on x86_64 - can you paste plain svg somewhere?

[dreamer]

Work on this feature started on branch po/fluid-1. Old version of this patch as distributed via ECE has some issues - we are going to use @realnc implementation instead (link) - it seems cleaner, was already converted to FluidSynth 2.1, and seems to be better tested. But we will need to do some tweaks anyway, as certain small design choices in there clash with our future plans.

[kcgen]

A recent platform-specific timer adjustment for fluidsynth: joncampbell123/dosbox-x@e00cf22

https://www.vogons.org/viewtopic.php?p=852446#p852446

[realnc]

A recent platform-specific timer adjustment for fluidsynth: joncampbell123/dosbox-x@e00cf22

https://www.vogons.org/viewtopic.php?p=852446#p852446

These are only used when fluidsynth is doing audio output itself. They are audio driver parameters. When rendering audio into a buffer (with fluid_synth_write_s16() in this case) and letting the dosbox mixer play the audio without creating a fluidsynth audio driver, these parameters have no effect whatsoever.

http://www.fluidsynth.org/api/index.html#UsingSynth

This is how it's done in dosbox-core. The fluidsynth patch that's been floating around for a while now for vanilla dosbox does not do this, and thus there these parameters are important.

[kcgen]

Thanks for the comparison @realnc, and good to know these adjustments won't be needed.

The approach of feeding dosbox's mixer with samples is win-win-win: fewer LOC to maintain, uses a single host-agnostic audio interface abstracted by SDL, and less runtime complexity and overhead.

[dreamer]

Initial, working version of FluidSynth integration can be tested on branch po/fluid-3 - at this point it is (almost) direct port from dosbox-core, but using our normal coding conventions, licensing info and SPDX identifier. Code was also moved to the recently created midi module, to avoid littering gui any more.

Testers: you need to compile it yourself. Our CI does not provide precompiled snapshots with FluidSynth integration (yet). FluidSynth 2.x is available in many distro repositories, but it's still missing from a few notable ones.

Do not ask me for support as of yet - you're on your own. Do not get married to new fluid settings as inherited from dosbox-core - we will change them (not sure exactly how yet).

I tested it on Ubuntu 20.04 and Windows 10 and it seemed to work fine, but code is not good enough quality-wise and we have no user documentation, so it won't be merged to master just yet.

[dreamer]

The first part of FluidSynth 2.x support was just merged via #539 :)

But I'm not closing this feature request just yet - we need to polish it a little bit, add more documentation, implement some missing bits, we have 1 small bug… but as of now, dosbox-staging finally has a built-in MIDI synth.

To testers: recreate your config file - there's a new fluidsynth section in. The current set of user-changeable settings is not final.

I am especially interested in learning from testers using wide range of SoundFonts:

• Is MIDI too loud or too quiet? Does it change with SoundFonts tested?
• Is it too CPU intensive? Is playback smooth?
• Is memory consumption appropriate? We're considering tweaking FluidSynth to load samples dynamically to lower the memory consumption (or maybe allowing users to change it)
• Are you missing something specific?
• What SoundFonts do you like and use?

If you're compiling the code yourself, then FluidSynth support should work on any OS.
If you're using our pre-compiled snapshot builds, then ATM only Windows builds have the feature enabled (fluidsynth 2.x library is missing from brew repo on macOS and from Ubuntu 18.04 repos, so we cannot provide pre-compiled packages on those OSes yet).

[arrowgent]

i think this is from SVN
source dosbox: https://launchpad.net/~i30817/+archive/ubuntu/dosbox-patched

##
#            fluid.driver: Driver to use with Fluidsynth, not needed under Windows. Available drivers depend on what Fluidsynth was compiled with
#                            Possible values: pulseaudio, alsa, oss, coreaudio, dsound, portaudio, sndman, jack, file, default.
#         fluid.soundfont: Soundfont to use with Fluidsynth. One must be specified.
#        fluid.samplerate: Sample rate to use with Fluidsynth.
#              fluid.gain: Fluidsynth gain.
#         fluid.polyphony: Fluidsynth polyphony.
#             fluid.cores: Fluidsynth CPU cores to use, default.
#           fluid.periods: Fluidsynth periods.
#        fluid.periodsize: Fluidsynth period size.
#            fluid.reverb: Fluidsynth use reverb.
#                            Possible values: no, yes.
#            fluid.chorus: Fluidsynth use chorus.
#                            Possible values: no, yes.
#   fluid.reverb,roomsize: Fluidsynth reverb room size.
#    fluid.reverb.damping: Fluidsynth reverb damping.
#      fluid.reverb.width: Fluidsynth reverb width. (.76)
#      fluid.reverb.level: Fluidsynth reverb level. (.57)
#     fluid.chorus.number: Fluidsynth chorus voices
#      fluid.chorus.level: Fluidsynth chorus level.
#      fluid.chorus.speed: Fluidsynth chorus speed.
#      fluid.chorus.depth: Fluidsynth chorus depth.
#       fluid.chorus.type: Fluidsynth chorus type. 0 is sine wave, 1 is triangle wave.
#                            Possible values: 0, 1.

i use a Roland SC-55 soundfont, i cant remember where i got it from
FluidR3_GM_sc-55.sf2 108424522 bytes

i dont have fluidsynth running all the time
i call it to start using a script then whatever application requires it
when the application exits so does fluidsynth.
ie:

/usr/bin/fluidsynth -a pulseaudio -m alsa_seq -i -l -s -p FluidSynth /usr/share/sounds/sf2/FluidR3_GM.sf2 &
/usr/games/dosbox &&

that said, i dont start fluidsynth with dosbox-staging
ive only done minor testing with dosbox-staging, sounds seem to be fine
there were a few games which music was not playing. i will try to figure out which games do/dont play
both by starting fluidsynth and by using static built-in fluidsynth

edit:
my mistake, im not using the fluidsynth patch. disreguard my testing

[arrowgent]

some dos midi software useful for testing?
some midi players could be useful for exclusively testing midi (no extra game/sfx/etc, and lightweight)

http://dosmid.sourceforge.net/

also GSPLAY 1.0
version gsplay1.zip or labelled 1.1
i dont see any public sources, although many respositories list it as "freeware" so i wont post a link here
not the same as gsplay v2.x free which is for windows

untested but others report using:
megamid dos

[grapeli]

Then launch the game with it.

#!/bin/bash
set -xeu

# Ensure we're running inside the script directory
cd "$(dirname "$0")"

# Find and preload the data files containing latency-sensitive
# MIDI data. The pattern gets both the HOMM2 original campaign
# and expansion pack.
find heroes2/data -iname 'hero*.agg' \
 | xargs -i dd if="{}" of=/dev/null bs=1M

# Launch dosbox
/path/to/dosbox-binary -conf -userconf dosbox.conf 

@kcgen
It reduces to about 4-6.
There are also two files in the startup sequence that are open for writing and are modified (heroes2.cfg).
Autosave.gm1 (106 kB), after opening and each time you click the hourglass (next day).

The 8GB flash drive (ext4) also fails this test (probably due to slow writing).

Regarding the configuration you proposed, I had to slightly modify it. The laptop, in addition to a weak HDD and CPU i5-450M, also has poor intel integrated graphics (Gen5) compatible only with OpenGL 2.1.
output = texture prevents scaling.

[sdl]
output = opengl
windowresolution = 1366x768

[cpu]
cycles = 60000

Here's a recording.
homm2.flac.txt
Likewise, most problems are at the beginning (when the I/O is loaded), then it's clean.

To be sure, I repeated this test in various configurations on tmpfs eight times. It is always 100% correct.

edit: I put autosave.gm1 and heroes2.cfg on tmps and made symbolic links. Even with that, it's not 100% correct.

[realnc]

Treating a single channel differently this way would be somewhat hacky solution in my opinion… But yeah, this is an option as well. I think putting FS into it's own thread will give a bit better results, but the proof is in the pudding as they say :)

I completely forgot: fsynth wouldn't be the exception. Munt should be treated the same way. Or any other MIDI synth, like BASSMIDI (if you decide to add that at some point.)

This makes sense even from a philosophical point of view. MIDI synths were always "asynchronous" devices back in the day. If you had an MT-32, a SoundCanvas, a Wave Blaster, or whatever else, the game would just send MIDI events over the MPU-401 port. The sound was rendered and played by those devices on their own. Even if the game went into an infinite loop, freezing completely, they would not stutter, hiccup, underrun, or anything like that. They would always play the sound cleanly, without any sound dropouts. Replicating that 100% async behavior makes sense.

[kcgen]

@grapeli,

Maybe there are other files (besides the big heroes*.agg) that load a bit of functional code prior to playing the MIDI sequences.

• Can you try symlinking just the two .agg files over to tmpfs?
• If you still get stutters then we know there are some other files that should be preloaded.
• You can then incrementally move more files over tmpfs, each time flushing and retesting, and hopefully narrow down exactly which files need to be preloaded to prevent the stuttering.

Other thoughts:
If your system is starved for memory (or if many applications are idle and soaking up memory or swap), then there will be a lot of roll-over eviction where earlier read data is quickly evicted to make way for newer reads (even if we're only talking about 250MB of game data). Memory starved systems also can behave very poorly when it comes to swapping out active application memory to disk; paging it back in will hard-block the application. There are lots of knobs to tune this behavior, but at this point it's no longer related to FluidSynth - so perhaps we could chat over Discord via PM.

Regarding slow CPU and GPU -- given the game plays flawless when moved to tmpfs, I think your CPU and GPU are fine. dosbox doesn't need much horse-power to hit 30000 cycles (my $30 raspberry pi plays HOMM2 flawlessly w/ MIDI at 30k cycles); so I think the problem is isolated to your system's memory and IO subsystems.

[grapeli]

@kcgen
I played HoMM2 a little (maybe too long). If I play it only with the music and sound turned off.
40,000 cycles (definitely) are not sufficiently smooth on my laptop in my opinion.
Broken Alliance I pass in 31 days at impossible (it's a very decent result).

I will still be testing.

edit: I built and tested the newest DOSBox-ECE-r4367 with fluidsynth-1.1.11. The sound is clear, smooth without any distortion. Same procedure with complete cleaning of the cache.
homm2.flac.txt
I normalized the sound. It is very quiet by default.
I have the impression that midi music (HoMM2) in dosbox-ece is delayed compared to dosbox-staging.

I don't know if this will explain anything, logs from perf top (approximate). Fluidsynth in dosbox staging without interpolate, reverb, chorus, etc. There are many differences.
The following immediately caught my eye:

0.20%  dosbox-staging [.] IO_ReadB
0.02%  dosbox-ece     [.] IO_ReadB

perf.top.dosbox-staging.txt
perf.top.dosbox-ece.txt

[grapeli]

The configuration is the same.

[midi]
mididevice = fluidsynth

[sblaster]
sbtype  = none

[gus]
gus = false

Cache cleaned every time.
Switch to fullscreen 3-4 times. Next.

dir
cd heroes2
dir > list
del list
exit

ece.flac.txt
staging.flac.txt

To make the difference visible, I increased the amplitude of the sound (-90dB).
How to explain this lack of fluidity in the sound source in dosbox-staging?
What is going on.

[kcgen]

Hi @grapeli,

ECE applies the FluidSynth patch as originally authored by @realnc, who explained that the patch makes FluidSynth write its generated audio out to the host OS using FluidSynth's own audio drivers for alsa / PulseAudio / CoreAudio (macOS) / Windows Audio. This audio output flow can happen asynchronously and in parallel to whatever dosbox is doing internally.

This differs from dosbox-staging's integration of @realnc's patch, which modified FluidSynth's audio output pathway to follow the same path as all the other emulated audio devices in dosbox, which is: raw PCM samples -> dosbox's mixer -> SDL -> host audio interface. All of this is driven in a 1ms synchronous and serialized loop.

Due to the lack of threading and output asynchronousity in the latter implementation, timing becomes absolutely critical. That is, all the PCM samples (from all the emulated audio sources) need to be computed on the spot in time to fill 1ms worth of buffer, then mixed, and then written to SDL.. all within 1ms. If there's any lag (including lag caused by other parts of the emulator not related to audio), there will be an audible gap in the audio.

[dreamer]

@kcgen I thought ECE applies an older version of the patch, still using (now being deprecated) FluidSynth 1.x?

@grapeli Regarding IO_ReadB - do you build dosbox-staging yourself? If yes, then remember to use -DNDEBUG and -O3 (as README.md suggests). Last time I checked we had no performance regressions compared to ECE, but well, this kind of regressions is hard to catch.

[kcgen]

@kcgen I thought ECE applies an older version of the patch, still using (now being deprecated) FluidSynth 1.x?

Good point -- maybe ECE's patch is prior to @realnc's updates?
It looks like it includes work from @kisai (Vogons.org forum).

I'm guessing it still using 1.x looking at configure.ac:

configure.ac:AC_CHECK_HEADER(fluidsynth.h, have_fluidsynth_h=yes, have_fluidsynth_h=no, )
configure.ac:AC_MSG_CHECKING(for FluidSynth support)
configure.ac:if test x$have_fluidsynth_lib = xyes -a x$have_fluidsynth_h = xyes ; then
configure.ac:  AC_DEFINE(C_FLUIDSYNTH,1)
configure.ac:  LIBS="$LIBS -lfluidsynth"
configure.ac:  AC_MSG_WARN([fluidsynth MIDI synthesis not available])

That said, the patch is definitely using (and requires) FluidSynth's own audio output driver as opposed to the channel -> mixer -> SDL pathway (as there is no channel callback or add_samples() call):

adriver = new_fluid_audio_driver(settings, synth);
if (!adriver) {
    LOG_MSG("MIDI:fluidsynth: Can't create audio driver");
    delete_fluid_synth(synth);
    delete_fluid_settings(settings);
    return false;
}

Regarding IO_ReadB: staging's IO handlers (approximately 270 of them) are stored in an STL template unordered map, which offers ~O(1) lookup times. This change let us drop the hard-coded 6 MiB array (of mostly blank values) to a mere ~8 KiB index.

IOBUS: Releasing 60 read and 53 write 8-bit port handlers
IOBUS: Releasing 33 read and 39 write 16-bit port handlers
IOBUS: Releasing 33 read and 35 write 32-bit port handlers
IOBUS: Handlers consumed 8432 total bytes

Speed-wise, these two are effectively the same: the 6 MiB doesn't fit in most L2 caches so can take a hit paging it in while the unordered map might have to lookup up two values instead of one - but has very high odds of fitting entirely in L1 cache. But in either case, these lookup times are dwarfed by the deliberate delay inserted into the IO handler code to prevent it going too fast and likely causing getting divide-by-zeros and such in the DOS program:

But there's another thing at play here: the unordered map is an STL template customized at compile time. When you build with profile instrumentation, then the map code itself becomes instrumented (which is great!) but at the same time, lookups become much slower due to that instrumentation. Where as the 6 MiB raw array isn't going to have that instrumentation. So that's going to act as "ballast weight" in the benchmark that disappears in a non-instrumented build. (and indeed, @dreamer and I both extensively benchmarked this change having the same concern and results were effectively identical).

[grapeli]

@kcgen I thought ECE applies an older version of the patch, still using (now being deprecated) FluidSynth 1.x?

Good point -- maybe ECE's patch is prior to @realnc's updates?
It looks like it includes work from @kisai (Vogons.org forum).

That said, the patch is definitely using (and requires) FluidSynth's own audio output driver as opposed to the channel -> mixer -> SDL pathway (as there is no channel callback or add_samples() call):

@kcgen
Exactly. The default midi config (ECE) was unchanged (fluid.gain=.6). Applying changes to the mixer has no effect (mixer fsynth 500, hence the signal level is very low).

@dreamer
Regarding IO_ReadB. There is no need to pay much attention to this. Perf top was made by hand, so it may have an error.

edit:
@kcgen
How can I correctly convert SDL PCM raw to wav.
I've tried so, but it doesn't sound right.

SDL_AUDIODRIVER=disk ./dosbox
ffmpeg -f u8 -ar 15734 -ac 1 -i sdlaudio.raw -ar 44100 -ac 1 out.wav
ffmpeg -f u8 -ar 8000 -ac 2 -i sdlaudio.raw -ar 44100 -ac 2 out.wav

https://hg.libsdl.org/SDL/file/c6ed576e4ba5/src/audio/disk/SDL_diskaudio.h
https://trac.ffmpeg.org/wiki/audio%20types
This gives a satisfactory effect.
ffmpeg -f s16le -ar 44100 -ac 2 -i sdlaudio.raw -ar 44100 -ac 2 out.wav
Needed when no sound card is present.

[grapeli]

I checked dosbox-staging under Windows 10 (same hardware). Configuration similar, soundfont used the same (FluidR3_GM.sf2), cold cache, HoMM2.
dosbox-staging-windows-x64-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)
dosbox-staging-windows-x86-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)

Then the x86-32 version under wine-5.19 under linux.
dosbox-staging-windows-x86-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)

The Linux version of dosbox-staging doesn't work (for me) properly with the built-in fluidsynth. I can rule out Alsa, I put it under Jack and it's the same.

[dreamer]

v0.76.0-alpha-871-g51084

WTF, this is a PR build, not a build from our master branch… We explicitly filter development builds to include only those from master branch… WTF, GitHub REST API?

(as for FluidSynth, functionally it's the same as 93c7d6f / current master branch)

The Linux version of dosbox-staging doesn't work (for me) properly with the built-in fluidsynth. I can rule out Alsa, I put it under Jack and it's the same.

@grapeli
We don't use Jack API, and SDL does not support it either. Can you add more details about what doesn't work? Do you test on a distro that packaged FluidSynth 2.x already? What soundfont do you use? I tested builds from the master branch on Fedora 33 Beta, using FluidR3 soundfont and it worked fine (aside of sound hiccups, that should be addressed by #640, which I will test next).

[grapeli]

@dreamer
SDL2 uses Jack (as opposed to SDL1). Of course.
SDL_AUDIODRIVER=jack ./dosbox

I'm testing a master with fluidsynth built from contrib. I wrote what sound font I use, I will write again - FluidR3_GM.sf2. I have sent over 5 samples showing what kind of problems it has with the sound output. I can send another one. It is no different. I am using Archlinux.

[dreamer]

Jack is not listed as a supported audio driver on SDL Wiki - the list says "it's partial", but I don't know if it's partial in regards to platforms or available backends.

@grapeli sorry, I haven't noticed you listed the soundfont and I thought you are reporting some new issue. Arch has Fluid 2.x, so you don't need to build from contrib - it's better to use the library as provided by your distro.

[grapeli]

@dreamer
sdl2 2.0.6
https://hg.libsdl.org/SDL/rev/eea7f98a37e4
I don't need Fluidsyth from the repository. It contains a number of redundant dependencies - libinstpatch, libsndfile, readline, dbus, systemd, ladspa, libpulse, etc. Dosbox-staging doesn't need them. Readline is "needed" for a fluidsynth server for an application that does not use it is redundant. Same with the rest. I don't use PulseAudio, etc...

With fluidsynth-2.1.5-2 from the repository, the sound is also interrupted.
homm2.flac.txt
Video with ffmpeg showspectrum filter.

[kcgen]

With fluidsynth-2.1.5-2 from the repository, the sound is also interrupted.

What tool are you using tho create this? This is extremely insightful; thanks for sharing this @grapeli!

Yet the build from contrib provides flawless sound (no interruptions) -- I wonder what the underlying reason it?

Even though our contrib is as lean and optimized as we can make it, the repo's library should (mostly) have dormant pathways to those unnecessary services. Perhaps the repo-build isn't optimized or retains debug instrumentation and checks?

Very interesting none the less.

[grapeli]

@kcgen
Very lame (no doubt it can be better).
SDL_AUDIODRIVER="alsa" AUDIODEV="hw:Loopback,0" ./dosbox

https://ffmpeg.org/ffmpeg-filters.html#showspectrum-1
ffmpeg -f alsa -c:a pcm_s16le -ac 2 -ar 44100 -i hw:2,1,0 -f x11grab -framerate 30000/1001 -video_size 1366x768 -i :0.0 -filter_complex "[0:a]showspectrum=s=200x768:orientation=horizontal,drawbox=w=iw:h=ih[vol]; [1:v]scale=1366:-1[scaled]; [scaled][vol]overlay[v]" -map "[v]" -map 0:a -c:v libx264 -pix_fmt yuv420p -qp 0 -preset ultrafast -g 60 /tmp/capture.01.mkv

https://ffmpeg.org/ffmpeg-filters.html#showvolume
ffmpeg -f alsa -c:a pcm_s16le -ac 2 -ar 44100 -i hw:2,1,0 -f x11grab -framerate 30000/1001 -video_size 1366x768 -i :0.0 -filter_complex "[0:a]showvolume=w=1366:p=0.6,drawbox=w=iw:h=ih[vol]; [1:v]scale=1366:-1[scaled]; [scaled][vol]overlay[v]" -map "[v]" -map 0:a -c:v libx264 -pix_fmt yuv420p -qp 0 -preset ultrafast -g 60 /tmp/capture.02.mkv

Yet the build from contrib provides flawless sound (no interruptions) -- I wonder what the underlying reason it?

No Linux version with built-in fluidsynth (with contrib, with the repository, with fluidlite) will produce the correct sound (with a cold cache).

All these audio interruptions (in my video) can be detected with ffprobe. You don't need to use audacity.
ffprobe -f lavfi -i amovie=/tmp/dosbox-staging-built-fluidsynth-homm2.mkv,astats=metadata=1:reset=1 -show_entries frame=pkt_pts_time:frame_tags=lavfi.astats.Overall.RMS_level,lavfi.astats.1.RMS_level,lavfi.astats.2.RMS_level -of csv=p=0
2.192993,-inf,-inf,-inf
...
2.235011,-inf,-inf,-inf

or
ffmpeg -i dosbox-staging-built-fluidsynth-homm2.mkv -af silencedetect=n=-70dB:d=0.01 -f null -

[grapeli]

I checked dosbox-staging under Windows 10 (same hardware). Configuration similar, soundfont used the same (FluidR3_GM.sf2), cold cache, HoMM2.
dosbox-staging-windows-x64-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)
dosbox-staging-windows-x86-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)

I tried to verify it again. Once again it turns out that my conclusions are too hasty. Sound is choppy in HoMM2 similar to Linux. It turns out that something else is heard on the headphones (speakers) and something else is obtained in the file with the captured sound (Ctrl-F6). I was fooled.
Win10 runs sluggishly on this hardware with hdd. It takes nearly 8 minutes to start up and restart the system. Tests under this system are not for me. I have no strength to fight him.

Then the x86-32 version under wine-5.19 under linux.
dosbox-staging-windows-x86-v0.76.0-alpha-871-g51084 - flawless sound (no interruptions)

The repeated wine test also fails.
I'm making a lot of confusion.

[dreamer]

@grapeli thanks for retest; I suggest you hold your horses a little bit until #640 will be in - that one improves the situation for me (so far tested with HoMM2 on Linux, using my old laptop).

Another thing, that seemingly improves the situation (but it might be placebo, I did no measurements yet) was changing to synth_threads = 2 (it introduces 1 new thread for FS synthesis, but I am not sure if it actually improves things, as we call fluidsynth in 1ms long chunks, which might be too small for multithreading to give us benefits at the moment).

Interestingly, I noticed that GOG release was toying with mixer prebuffer value for this game…

And yeah - your experience with sound issues happening outside of dosbox-staging (only when testing with headphones but not when recording) are similar to mine.

[kcgen]

your experience with sound issues happening outside of dosbox-staging (only when testing with headphones but not when recording) are similar to mine.

@grapeli, @dreamer - this is so subtle and tricky; but fantastic you are both reproducing it. Wish we could get to the bottom of what's sitting between our output stream to SDL (the recording) and you headphones.

Maybe its time for contrib/static-sdl2: lean and mean, to isolate any repo-side build issues.

[grapeli]

Midi sound interruptions with dosbox-staging built-in fluidsynth occur:
-only from hdd, or a slow flash drive, flash card
-only with a cold cache (and only then)
-most often when performing an interaction in the form of clicking on the touchpad, pressing the mouse button, pressing the keyboard
-when switching to fullscreen (and vice versa), switching to another application window (and back).

Never when the HoMM2 directory is mounted on tmpfs.
Never with external fluidsynth.
I also tested dosbox-ece and dosbox-x with fluidsynth built in. The sound is smooth without any distortion.

I tested dosbox-staging in various variants. With static SDL2, with synth_threads=2. With different audio outputs (dummy, alsa, jack).
With different linux kernels. In various configurations from CONFIG_PREEMPT_NONE to CONFIG_PREEMPT, from CONFIG_HZ_100 to CONFIG_HZ_1000. From distribution (archlinux) to BMQ, from LTS, longterm to mainline. The configuration and kernel version have no effect on this.

I use statically built SDL2 a lot. For example, a niche WM - notion, I use needs a patch to make SDL2 work properly in full screen. edit: A broken fullscreen in SDL2 was finally fixed five days ago (under notion, xmonad and probably other WM).

[dreamer]

With #734 finished, I think it's time to close this issue :)

FluidSynth 2.x integration is not perfect yet, but it already provides the best out-of-the-box experience compared to any MIDI solution I tested with any DOSBox fork so far :)

We picked good defaults, support soundfont lookup in standard directories, gain configuration that's easy to correlate to a specific font, have updated documentation to reflect the changes, and more.

To enable FluidSynth MIDI, user needs to set:

[midi]
mididevice = fluidsynth
mpu401 = intelligent

[fluidsynth]
soundfont = filename.sf2  # file will be looked in current working dir, then
                          # in 'soundfonts' dir in config dir, and a number
                          # of global directories that might store .sf2 files.

# alternatively:
soundfont = ~/my-soundfonts/foobar.sf2  # relative, absolute paths and tilde-expansion
                                        # is supported

# alternatively:
soundfont = quiet-font.sf2 150  # gain is adjusted to 150%
soundfont = loud-font.sf2 30    # gain is reduced to 30%

Empty soundfonts dir is created for the user automatically, so it's clear where to store fonts for DOS games. Or they can be stored with the game itself (assuming user changed CWD to game directory).

The volume of FluidSynth MIDI channel (FSYNTH) can be adjusted via DOS mixer command (the same way as every other built-in emulated audio channel).

DOSBox Staging will look for default.sf2 file by default, as this is the default name used by FluidSynth - on Linux systems it's usually a symbolic link to the newest version of Fluid_R3.sf2 (if it's installed).

@kcgen, @grapeli, @realnc, @nemo93, @bluddy
Thank you everyone for testing, let's continue improving the feature in #735 :)