State of the runtime

[p0nce]

How does the using-runtime plug-in fare in real hosts?

• Linux single instance (see if using-runtime can be loaded in one host)
• Linux multiple instance (same but instantiating twice that plugin)
• Update tests\using-runtime outdated parts (dub.json, plugin.json)
• Windows single instance
• Windows multiple instance
• macOS single instance (blocked by mac + latest process + msencode example => crash #291)
• macOS multiple instance (blocked by mac + latest process + msencode example => crash #291)

[p0nce]

OK, this mostly crash everywhere for now.
(EDIT: was probably the exception passing through, this was optimistic in retrospect)

[p0nce]

on macOS test works but crash Live on load

[p0nce]

Confirmation that on OSX, __gshared variables are shared across multiple version of the same dynlib.

• on OSX, the Runtime cannot be initialized twice in different plugin instantiation, since there is something shared about them. A cop-out solution is to not terminate the Runtime on OSX.
  A similar dynamic lead us to leak every shared library open within Dplug plug-ins on OSX.

• try to terminate the runtime on dynlib unload. (EDIT: works! Every OSX leak can disappear this way!)

[p0nce]

Works on some linuxes.

Reported by @0xEAB:

Process 8118 stopped
* thread #10, name = 'reaper', stop reason = signal SIGSEGV: invalid address (fault address: 0x10)
    frame #0: 0x00007fffd447bd7f libdruntime-ldc-shared.so.79`_D4core6thread7suspendFNbCQyQv6ThreadZb + 31
libdruntime-ldc-shared.so.79`_D4core6thread7suspendFNbCQyQv6ThreadZb:
->  0x7fffd447bd7f <+31>: cmpq   $0x0, 0x10(%r12)
    0x7fffd447bd85 <+37>: je     0x7fffd447be6a            ; <+266>
    0x7fffd447bd8b <+43>: leaq   0x48(%r12), %r14
    0x7fffd447bd90 <+48>: movq   0x66329(%rip), %rbx
(lldb) bt
* thread #10, name = 'reaper', stop reason = signal SIGSEGV: invalid address (fault address: 0x10)
  * frame #0: 0x00007fffd447bd7f libdruntime-ldc-shared.so.79`_D4core6thread7suspendFNbCQyQv6ThreadZb + 31
    frame #1: 0x00007fffd447c01a libdruntime-ldc-shared.so.79`thread_suspendAll + 74
    frame #2: 0x00007fffd448370a libdruntime-ldc-shared.so.79`_D2gc4impl12conservativeQw3Gcx11fullcollectMFNbbZm + 90
    frame #3: 0x00007fffd4484ee6 libdruntime-ldc-shared.so.79`_D2gc4impl12conservativeQw3Gcx10smallAllocMFNbhKmkZPv + 118
    frame #4: 0x00007fffd4481bf4 libdruntime-ldc-shared.so.79`_D2gc4impl12conservativeQw14ConservativeGC__T9runLockedS_DQCeQCeQCcQCnQBs12mallocNoSyncMFNbmkKmxC8TypeInfoZPvS_DQEgQEgQEeQEp10mallocTimelS_DQFiQFiQFgQFr10numMallocslTmTkTmTxQCzZQFcMFNbKmKkKmKxQDsZQDl + 100
    frame #5: 0x00007fffd4484065 libdruntime-ldc-shared.so.79`_DThn16_2gc4impl12conservativeQw14ConservativeGC6mallocMFNbmkxC8TypeInfoZPv + 53
    frame #6: 0x00007fffd447b577 libdruntime-ldc-shared.so.79`thread_attachThis + 39
    frame #7: 0x00007fffee9c8f02 No Name Audio Runtime User.so`_D5dplug4core7runtime20ScopedRuntimeSection5enterMFNbZv(this=0x00007fffee98ae00) at runtime.d:225
    frame #8: 0x00007fffee9a307e No Name Audio Runtime User.so`_D5dplug4core7runtime__T14runtimeSectionTDFNbKC4main10HeapObjectZiZQBrFNbNiQBiZ12internalFuncFNbQCdKQCcZi(fun=int delegate(ref main.HeapObject) nothrow @system @ 0x00007fffabffce70, _param_1=0x0000000001010f08) at main.d:77
    frame #9: 0x00007fffee98aee5 No Name Audio Runtime User.so`_D5dplug4core7runtime__T14runtimeSectionTDFNbKC4main10HeapObjectZiZQBrFNbNiQBiZ14ManualDelegate6opCallMFNbNiKQClZi(this=<unavailable>, _param_0=<unavailable>) at runtime.d:96
    frame #10: 0x00007fffee98ac5c No Name Audio Runtime User.so`_D4main17RuntimeTestPlugin5resetMFNbNidiiiZv(this=0x000000000100fe00, sampleRate=44100, maxFrames=128, numInputs=2, numOutputs=2) at main.d:101
    frame #11: 0x00007fffee9b02a6 No Name Audio Runtime User.so`_D5dplug6clientQh6Client13resetFromHostMFNbNidiiiZv(this=0x000000000100fe00, sampleRate=44100, maxFrames=128, numInputs=2, numOutputs=2) at client.d:640
    frame #12: 0x00007fffee9a6cca No Name Audio Runtime User.so`_D5dplug3vst6client9VSTClient15processMessagesMFNbNiZv(this=0x0000000001014cd0) at client.d:848
    frame #13: 0x00007fffee9a7993 No Name Audio Runtime User.so`_D5dplug3vst6client9VSTClient22processDoubleReplacingMFNbNiPPdQdiZv(this=0x0000000001014cd0, inputs=0x00007fffa00f2940, outputs=0x00007fffa00f2950, sampleFrames=512) at client.d:950
    frame #14: 0x00007fffee9a5135 No Name Audio Runtime User.so`processDoubleReplacingCallback(effect=0x0000000001014ce0, inputs=0x00007fffa00f2940, outputs=0x00007fffa00f2950, sampleFrames=512) at client.d:1071
    frame #15: 0x00000000008334bf reaper`___lldb_unnamed_symbol6374$$reaper + 11247
    frame #16: 0x00000000006db9c8 reaper`___lldb_unnamed_symbol4786$$reaper + 3464
    frame #17: 0x00000000006c906e reaper`___lldb_unnamed_symbol4688$$reaper + 350
    frame #18: 0x0000000000662044 reaper`___lldb_unnamed_symbol4091$$reaper + 9972
    frame #19: 0x0000000000667527 reaper`___lldb_unnamed_symbol4092$$reaper + 791
    frame #20: 0x0000000000668827 reaper`___lldb_unnamed_symbol4094$$reaper + 1511
    frame #21: 0x000000000063bd87 reaper`___lldb_unnamed_symbol3899$$reaper + 1671
    frame #22: 0x00007ffff64da51b libSwell.so`___lldb_unnamed_symbol21$$libSwell.so + 11
    frame #23: 0x00007ffff7bbd6db libpthread.so.0`start_thread + 219
    frame #24: 0x00007ffff6a9d88f libc.so.6`__GI___clone at clone.S:95

Apart from the crash, a problem is that this is a shared runtime. Since the crash happen in "libdruntime-ldc-shared.so".

When trying not to link with a shared runtime (-static -shared), link errors happen:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/usr/bin/ld: final link failed: Nonrepresentable section on output

Upon analysis it seem the static runtime are built with PIC, so it would just be a matter of selecting them with -default-lib?

[p0nce]

Because of https://issues.dlang.org/show_bug.cgi?id=18456 when built with DMD the runtime will leak instead.
This is meant to make it build, the only downsides is that GC object destructors won't be called by the GC. But you shouldn't rely on it already so...

[p0nce]

On Linux it seems plug-in build against the shared runtime! This is probably bad for redistribution.

[p0nce]

An explanation for the shared runtime? dlang/dub#647

[p0nce]

On Linux, it seems we have a shared runtime with LDC and a static runtime with DMD (source = @abaga129 )

[p0nce]

Should be fixed if #391 is solved with a static Linux runtime.

[p0nce]

Actually linking with a static runtime in a shared library probably won't work because -fPIC. And then there is no solution on Linux and we'd rather abandon this idea of runtime sections. It doesn't make sense to make those Mac and Windows only.

[0xEAB]

Actually linking with a static runtime in a shared library probably won't work because -fPIC.

so, how does this work on macOS then?

[p0nce]

On macOS all code is built with -fPIC

[abaga129]

I built the using-runtime test and ran it on a fresh install of Ubuntu 18.04.

So far I've only tested on Reaper, but these are my results.
Built with DMD the plugin loads correctly the first time but crashes the host after being loaded a second time.

Built with ldc2 using the updated flags in dub.json the plugin does not load. Running GDB with Reaper gives me the following exception while scanning

swell: dlopen() failed: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/reker/.vst/libusing-runtime.so

[p0nce]

Mmmm interesting the LDC2 things should affect current plugins too.
Other than that, I feel like we should rather give up for #292

[abaga129]

It could be that the GLIBC that I built against was different than what is available on Ubuntu 18.04. I built the plugin on Arch Linux and transfered it over. It doesn't explain why the DMD build works though.
On my Arch install it has glibc 2.29 while on Ubuntu 18.04 the current glibc is 2.27.

I'm okay with abandoning runtime support though.

[p0nce]

Mmm your last point is interesting, the build machine should be not at the edge of Linux ABIs then

[p0nce]

Closed because it seems everytime part of the plugin has enabled the D runtime it created problems, this won't be a priority before a long time.