Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RunTime Error: "Create Jack audio port. Failed to create the audio driver. free(): double free detected in tcache 2. #613

Closed
jofemodo opened this issue Jan 22, 2020 · 18 comments
Labels

Comments

@jofemodo
Copy link

FluidSynth version

Version: v2.1.0 or latest from master (working fine on 2.0.9)

Describe the bug

When starting fluidsynth with:

/usr/local/bin/fluidsynth -p fluidsynth -a jack -m jack -g 1 -j -o synth.midi-bank-select=mma -o synth.cpu-cores=3 -o synth.polyphony=64

I get:

Create Jack audio port.
Failed to create the audio driver.
free(): double free detected in tcache 2

Expected behavior

Get fluidsynth prompt. No error.

Steps to reproduce

The error is quite random. Not 100% reproducible.
But it's easy to reproduce on fresh boot.

Additional context

Platform: Raspberry Pi, running custom Raspbian (ZynthianOS)

@jofemodo jofemodo added the bug label Jan 22, 2020
@derselbst
Copy link
Member

Could you please provide a stack trace? Ideally by compiling fluidsynth with

cmake -Denable-ubsan=1 -Denable-debug=1 ..

(requires UndefinedBehaviorSanitizer and AddressSanitizer to be installed).

@jofemodo
Copy link
Author

When running the new binary I get:

==3769==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.

@jofemodo
Copy link
Author

Note: I installed the libubsan0 & libasan4 packages.

@jofemodo
Copy link
Author

OK! Got it!

export LD_PRELOAD=usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so; 
root@zynthian:/zynthian/zynthian-sw/fluidsynth/build# /usr/local/bin/fluidsynth -p fluidsynth -a jack -m jack -g 1 -j -o synth.midi-bank-select=mma -o synth.cpu-cores=3 -o synth.polyphony=64
ERROR: ld.so: object 'usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
==4648==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
root@zynthian:/zynthian/zynthian-sw/fluidsynth/build# export LD_PRELOAD=/usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so; 
root@zynthian:/zynthian/zynthian-sw/fluidsynth/build# /usr/local/bin/fluidsynth -p fluidsynth -a jack -m jack -g 1 -j -o synth.midi-bank-select=mma -o synth.cpu-cores=3 -o synth.polyphony=64
FluidSynth runtime version 2.1.0
Copyright (C) 2000-2019 Peter Hanappe and others.
Distributed under the LGPL license.
SoundFont(R) is a registered trademark of E-mu Systems, Inc.

fluidsynth: debug: Using 'jack' midi driver
fluidsynth: debug: Using 'jack' audio driver
fluidsynth: jack midi autoconnect "system:midi_capture_1" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "ttymidi:MIDI_in" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "a2j:Midi Through [14] (capture): Midi Through Port-0" to "fluidsynth-midi:midi_00"
fluidsynth: error: Failed to create Jack audio port
fluidsynth: jack midi autoconnect "system:midi_capture_1" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "ttymidi:MIDI_in" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "a2j:Midi Through [14] (capture): Midi Through Port-0" to "fluidsynth-midi:midi_00"
Failed to create the audio driver
=================================================================
==4810==ERROR: AddressSanitizer: heap-use-after-free on address 0xb3102298 at pc 0xb65710ec bp 0xbeb9bc7c sp 0xbeb9bc74
READ of size 4 at 0xb3102298 thread T0
    #0 0xb65710eb in fluid_jack_client_close /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:530
    #1 0xb6576a2b in delete_fluid_jack_midi_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:887
    #2 0xb66c9613 in delete_fluid_midi_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_mdriver.c:192
    #3 0x1678f in main /zynthian/zynthian-sw/fluidsynth/src/fluidsynth.c:1053
    #4 0xb5eb5717 in __libc_start_main /build/glibc-FUvrFr/glibc-2.28/csu/libc-start.c:308

0xb3102298 is located 8 bytes inside of 16-byte region [0xb3102290,0xb31022a0)
freed by thread T0 here:
    #0 0xb6a2f887 in __interceptor_free (/usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so+0xe1887)
    #1 0xb6597b8b in fluid_free /zynthian/zynthian-sw/fluidsynth/src/utils/fluid_sys.c:226
    #2 0xb656d173 in new_fluid_jack_client /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:317
    #3 0xb6571aa3 in new_fluid_jack_audio_driver2 /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:596
    #4 0xb657179b in new_fluid_jack_audio_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:571
    #5 0xb66c7a43 in new_fluid_audio_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_adriver.c:314
    #6 0x164c3 in main /zynthian/zynthian-sw/fluidsynth/src/fluidsynth.c:985
    #7 0xb5eb5717 in __libc_start_main /build/glibc-FUvrFr/glibc-2.28/csu/libc-start.c:308

previously allocated by thread T0 here:
    #0 0xb6a2fbbb in __interceptor_malloc (/usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so+0xe1bbb)
    #1 0xb6597b1f in fluid_alloc /zynthian/zynthian-sw/fluidsynth/src/utils/fluid_sys.c:197
    #2 0xb656c1b7 in new_fluid_jack_client /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:201
    #3 0xb657672b in new_fluid_jack_midi_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:864
    #4 0xb66c91ff in new_fluid_midi_driver /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_mdriver.c:155
    #5 0x158db in main /zynthian/zynthian-sw/fluidsynth/src/fluidsynth.c:839
    #6 0xb5eb5717 in __libc_start_main /build/glibc-FUvrFr/glibc-2.28/csu/libc-start.c:308

SUMMARY: AddressSanitizer: heap-use-after-free /zynthian/zynthian-sw/fluidsynth/src/drivers/fluid_jack.c:530 in fluid_jack_client_close
Shadow bytes around the buggy address:
  0x36620400: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa
  0x36620410: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa
  0x36620420: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa
  0x36620430: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa
  0x36620440: fa fa fd fd fa fa fd fd fa fa 04 fa fa fa fd fd
=>0x36620450: fa fa fd[fd]fa fa 01 fa fa fa 00 fa fa fa fd fa
  0x36620460: fa fa 00 04 fa fa fd fa fa fa fd fa fa fa 00 04
  0x36620470: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 04
  0x36620480: fa fa 00 04 fa fa fd fa fa fa 00 fa fa fa 00 00
  0x36620490: fa fa fd fa fa fa 00 00 fa fa fd fa fa fa 04 fa
  0x366204a0: fa fa 05 fa fa fa 05 fa fa fa 00 03 fa fa fd fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==4810==ABORTING

@derselbst
Copy link
Member

Great, thanks for the log, that's very helpful!

The following happens: For some reason, fluidsynth is unable to open an audio port at your jack server (i.e. jack_port_register() fails). Since 2.1.0, this case is checked and an error message is printed (Failed to create Jack audio port). Versions 2.0.9 and before have not checked this case, which I would expect to lead to a NULL dereference later on and crash as well.

Anyway, the audio driver handles this case by cleaning up the jack client. Unfortunately, this jack client is also being used by the jack midi driver which has been created earlier as well. And when that midi driver frees its resources, it wants to free the client again.

This is not trivial to fix. I need some time to think about this.

@jofemodo Pls, could you confirm that you have absolutely no issues with 2.0.9? So it processes all MIDI events and produces sound without any issues?

@jofemodo
Copy link
Author

jofemodo commented Jan 22, 2020

Yes, AFAIK, It works perfectly fine with 2.0.9

Thanks!

@derselbst
Copy link
Member

@jofemodo I've just pushed a fix to master. It fixes the double free for me locally. Feel free to test.

Not sure why 2.1.0 is unable to connect to the port(s) though. Our jack driver hasn't really changed. And Jack should at least print an error. You could try to start fluidsynth without the -j option and see whether it makes a difference. Alternatively, try passing -o synth.lock-memory=0 when starting fluidsynth.

@derselbst
Copy link
Member

@jofemodo Any news on this?

@derselbst
Copy link
Member

I'm considering this to be fixed by 5070fe8. Thanks for the report.

@jofemodo
Copy link
Author

jofemodo commented Jul 8, 2020

Hi @derselbst !

Sorry for the radio silence ... finally i got some time for checking the latest versions of fluidsynth, but the error is still there on 2.1.0, and also on 2.1.3. Same error. So we keep running 2.0.9 on zynthian...

Thanks!

@derselbst
Copy link
Member

There was a NULL dereference. I fixed it. Pls try again. If it crashes, pls provide me the same detailed log as above.

As I still have no clue, why fluidsynth is unable to create a jack port, you might try starting jackd manually with the --verbose flag and share the logs.

If you don't have time to test it, just let me know, as I'm planning to release 2.1.4 soon.

@derselbst derselbst reopened this Jul 9, 2020
@jofemodo
Copy link
Author

jofemodo commented Jul 10, 2020

It still fails:

root@zynthian:~# export LD_PRELOAD=/usr/lib/gcc/arm-linux-gnueabihf/8/libasan.so;
root@zynthian:~# /usr/local/bin/fluidsynth -p fluidsynth -a jack -m jack -g 1 -j -o synth.midi-bank-select=mma -o synth.cpu-cores=3 -o synth.polyphony=64
FluidSynth runtime version 2.1.4
Copyright (C) 2000-2020 Peter Hanappe and others.
Distributed under the LGPL license.
SoundFont(R) is a registered trademark of E-mu Systems, Inc.

fluidsynth: jack midi autoconnect "system:midi_capture_1" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "ttymidi:MIDI_in" to "fluidsynth-midi:midi_00"
fluidsynth: jack midi autoconnect "a2j:Midi Through [14] (capture): Midi Through Port-0" to "fluidsynth-midi:midi_00"
fluidsynth: error: Failed to create Jack audio port 'right'
Failed to create the audio driver

Compiled with asan, but not address error dumped.
It seems the same error, failing 100% on first execution after fresh boot.

Regards,

@derselbst
Copy link
Member

Thank you for testing. Good to see that it doesn't crash anymore.

Unfortunately, I have no clue why fluidsynth fails to create the audio port. I would expect jack to write an error message, when a call to jack_port_register() fails. And as you can see from the source code, our jack driver basically hasn't changed from 2.0.9 to 2.1.0:

$ git diff v2.0.9 v2.1.0
diff --git a/src/drivers/fluid_jack.c b/src/drivers/fluid_jack.c
index 60503e2f..31706ae8 100644
--- a/src/drivers/fluid_jack.c
+++ b/src/drivers/fluid_jack.c
@@ -263,7 +263,7 @@ new_fluid_jack_client(fluid_settings_t *settings, int isaudio, void *driver)
     }
 
     /* tell the lash server our client name */
-#ifdef LASH_ENABLED
+#ifdef HAVE_LASH
     {
         int enable_lash = 0;
         fluid_settings_getint(settings, "lash.enable", &enable_lash);
@@ -273,7 +273,7 @@ new_fluid_jack_client(fluid_settings_t *settings, int isaudio, void *driver)
             fluid_lash_jack_client_name(fluid_lash_client, name);
         }
     }
-#endif /* LASH_ENABLED */
+#endif /* HAVE_LASH */
 
     client_ref->server = server;        /* !! takes over allocation */
     server = NULL;      /* Set to NULL so it doesn't get freed below */
@@ -328,7 +328,7 @@ fluid_jack_client_register_ports(void *driver, int isaudio, jack_client_t *clien
     char name[64];
     int multi;
     int i;
-    int jack_srate;
+    unsigned long jack_srate;
     double sample_rate;
 
     if(!isaudio)
@@ -491,10 +491,10 @@ fluid_jack_client_register_ports(void *driver, int isaudio, jack_client_t *clien
 
     fluid_settings_getnum(settings, "synth.sample-rate", &sample_rate);
 
-    if((int)sample_rate != jack_srate)
+    if((unsigned long)sample_rate != jack_srate)
     {
         FLUID_LOG(FLUID_INFO, "Jack sample rate mismatch, adjusting."
-                  " (synth.sample-rate=%lu, jackd=%lu)", (int)sample_rate, jack_srate);
+                  " (synth.sample-rate=%lu, jackd=%lu)", (unsigned long)sample_rate, jack_srate);
         fluid_settings_setnum(settings, "synth.sample-rate", jack_srate);
     }

So, since I see no evidence for fluidsynth's failure, I'm closing this ticket.

@jofemodo
Copy link
Author

I can see the code ... yes ... but ...

  • when i compile 2.0.9, it works like a charm.
  • when compiling 2.1.x, it fails.

These are facts too. Same machine, same build environment, same run-time environment, nothing changes except fluidsynth version.

Anyway, we can stay on 2.0.9 and let's see if future versions solves the problem

Thanks!

@derselbst
Copy link
Member

I understand that this situation is not satisfying, but if jack_port_register() returns NULL, there is not much we can do I'm afraid. Esp. when its usage hasn't changed. Again, some ideas that you could try:

  • start jackd manually from commandline with the --verbose flag to get some meaningful log messages
  • start fluidsynth without the -j option
  • pass -o synth.lock-memory=0 when starting fluidsynth
  • pass -o audio.jack.multi=1 when starting fluidsynth
  • double check with fluidsynth --version that the "runtime version" and "executable version" are the same for both 2.0.9 and 2.1.x
  • verify with ldd fluidsynth | grep jack that 2.0.9 and 2.1.x both use the same libjack.so
  • debug the call to jack_port_register() to see where it fails
  • ask at fluidsynth's mailing list whether anybody has observed this behaviour
  • ask at jackaudio upstream for advice

@lafleurdeboum
Copy link

Hello ; I found the same error on a Raspberry Pi zero using fluidsynth 2.1.5-2 in Arch :

fluidsynth: error: Failed to create Jack audio port 'left'

fluidsynth still runs, but the audio connection is not made (the midi autoconnection seems to work fine). If I connect it manually with njconnect, the sound still doesn't come out. I just wanted to mention that this happens only if the MIDI device is plugged in before boot. If I plug it in after fluidsynth has started, everything runs fine.

I'm using the following commands :

/usr/bin/jackd -d alsa -d hw:sndrpihifiberry -n2 -p128 -Xraw
/usr/bin/fluidsynth -is -a jack -j -m jack -o synth.polyphony 64 -f /usr/share/soundfonts/fluidsynth.command /usr/share/soundfonts/default.sf2

I also don't get this error when using alsa sequencer (leaving -X option out of jackd, and setting -m alsa_seq in fluidsynth).

I will try your other tests when I get the time to ! Please ask if you need some other log.

Here is my JACK2 (version 1.9.14-1) log :

Sep 30 18:58:57 zero jackd[272]: JACK server starting in realtime mode with priority 10
Sep 30 18:58:57 zero jackd[272]: self-connect-mode is "Don't restrict self connect requests"
Sep 30 18:58:58 zero jackd[272]: audio_reservation_init
Sep 30 18:58:58 zero jackd[272]: Acquire audio card Audio1
Sep 30 18:58:58 zero jackd[272]: creating alsa driver ... hw:sndrpihifiberry|hw:sndrpihifiberry|128|2|48000|0|0|nomon|swmeter|-|32bit
Sep 30 18:58:58 zero jackd[272]: ALSA: Cannot open PCM device alsa_pcm for capture. Falling back to playback-only mode
Sep 30 18:58:58 zero jackd[272]: configuring for 48000Hz, period = 128 frames (2.7 ms), buffer = 2 periods
Sep 30 18:58:58 zero jackd[272]: ALSA: final selected sample format for playback: 32bit integer little-endian
Sep 30 18:58:58 zero jackd[272]: ALSA: use 2 periods for playback
Sep 30 18:58:58 zero jackd[272]: scan: added port hw:0,0,0 in-hw-0-0-0-Impact-GX61-MIDI-1
Sep 30 18:58:58 zero jackd[272]: scan: added port hw:0,0,1 in-hw-0-0-1-Impact-GX61-MIDI-2
Sep 30 18:58:58 zero jackd[272]: scan: added port hw:0,0,0 out-hw-0-0-0-Impact-GX61-MIDI-1
Sep 30 18:58:58 zero systemd[244]: Started JACK server using hifiberry.conf profile.
Sep 30 18:58:58 zero jackd[272]: scan: opened port hw:0,0,0 in-hw-0-0-0-Impact-GX61-MIDI-1
Sep 30 18:58:58 zero jackd[272]: scan: opened port hw:0,0,1 in-hw-0-0-1-Impact-GX61-MIDI-2
Sep 30 18:58:59 zero jackd[272]: scan: opened port hw:0,0,0 out-hw-0-0-0-Impact-GX61-MIDI-1
Sep 30 18:59:05 zero jackd[272]: JackEngine::XRun: client = fluidsynth-midi was not finished, state = Running
Sep 30 18:59:05 zero jackd[272]: JackAudioDriver::ProcessGraphAsyncMaster: Process error
Sep 30 18:59:05 zero jackd[272]: JackEngine::XRun: client = fluidsynth-midi was not finished, state = Triggered
[... many XRuns ...]
Sep 30 18:59:06 zero jackd[272]: JackAudioDriver::ProcessGraphAsyncMaster: Process error
Sep 30 18:59:06 zero jackd[272]: JackGraphManager::Connect already connected port_src = 3 port_dst = 6
Sep 30 18:59:06 zero jackd[272]: JackEngine::XRun: client = fluidsynth-midi was not finished, state = Triggered
Sep 30 18:59:06 zero jackd[272]: JackAudioDriver::ProcessGraphAsyncMaster: Process error
Sep 30 18:59:06 zero jackd[272]: JackEngine::XRun: client = fluidsynth-midi was not finished, state = Triggered
Sep 30 18:59:06 zero jackd[272]: JackAudioDriver::ProcessGraphAsyncMaster: Process error
Sep 30 18:59:06 zero jackd[272]: JackGraphManager::Connect already connected port_src = 4 port_dst = 6
Sep 30 18:59:06 zero jackd[272]: CheckSize error size = 32 Size() = 12
Sep 30 18:59:06 zero jackd[272]: CheckRead error
Sep 30 18:59:06 zero jackd[272]: CheckSize error size = -1 Size() = 4
Sep 30 18:59:06 zero jackd[272]: CheckRead error
Sep 30 18:59:06 zero jackd[272]: CheckSize error size = 0 Size() = 12
Sep 30 18:59:06 zero jackd[272]: CheckRead error
Sep 30 18:59:59 zero jackd[272]: JackEngine::XRun: client = fluidsynth-midi was not finished, state = Triggered
Sep 30 18:59:59 zero jackd[272]: JackAudioDriver::ProcessGraphAsyncMaster: Process error

@derselbst
Copy link
Member

The original bug was about a double free. This has been fixed, as you confirm "fluidsynth still runs". That the audio port connection fails is a different problem. Thus I would appreciate if you open a new issue and share your findings there. Perhaps we'll need a joint investigation with the guys from Jack, which will be easier in a new, dedicated issue.

@lafleurdeboum
Copy link

OK ! Yes I can see I rushed into this issue a bit quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants