add support for linux dynamic dll loading

[WalterBright]

First stab at supporting dynamically loadable shared libraries under Linux. It's magical, because (thanks to @dawgfoto's ideas) you don't have to do anything other than use the Linux API to load/unload DLLs. They initialize themselves!

(Currently won't work if you try to load DLLs at the same time from different threads, needs some synchronization.)

Also needs some thought towards what should happen if an exception is thrown during module construction/destruction.

Once we get thoroughly satisfied that this is right, we can extend it to the other platforms.

[WalterBright]

Looks like something is going wrong in the thread shutdown. I'll look into it tomorrow.

[alexrp]

Wait what? Debug leftover?

[WalterBright]

Running unit tests is not really part of initialization. I moved the call to line 567.

[alexrp]

Module-related code is beyond me; cc @dawgfoto

[MartinNowak]

Among other things we cannot initialize a shared library for another thread but the loading one. This means we must not use one globally shared data structure (_static_dsos). Instead it should contain only the linked libraries. For the dynamically loaded libraries we should use a thread local data structure.

[MartinNowak]

While handling rt.dmain2's constructor specially works, we could simply run rt_init before the first module constructor (here) and rt_term after the last one. I think that would be a cleaner solution.

[complexmath]

Will this really work the way we want? Since there's no dependency tree here, it seems like this static dtor will be run in a random order with the others, and so some static dtors will be run after the GC and runtime are completely shut down.

[WalterBright]

In minfo.d, I added code specifically to run this ctor first, and this dtor last.

[MartinNowak]

Just call rt_init within the first call to _d_dso_registry and rt_term when the last library unregisters. This is dead simple and you can scratch all the code to treat rt.dmain2 specially.

[WalterBright]

Among other things we cannot initialize a shared library for another thread but the loading one. This means we must not use one globally shared data structure (_static_dsos). Instead it should contain only the linked libraries. For the dynamically loaded libraries we should use a thread local data structure.

I know _static_dsos[] needs to be protected with a mutex. Other than that, I believe the design is sound.

[WalterBright]

While handling rt.dmain2's constructor specially works, we could simply run rt_init before the first module constructor (here) and rt_term after the last one. I think that would be a cleaner solution.

You may be right. But I'd like to get everything working first, and then we can look at it again and perhaps refactor it.

[braddr]

This whole area is in need of a comprehensive set of tests. Have you been building them or sticking to ad-hoc tests?

[WalterBright]

A great deal of it is tested already with the existing suite.

For the more specific tests for loading/unloading, yes I have some, and I've been working up in parallel a guide to writing such DLLs. But I was going to ask for your help in integrating them into the auto tester.

[WalterBright]

Hopefully got the std.concurrency bug fixed. Turns out that atexit() runs before the .ctors, which caused the mutexes used by the GC to be prematurely destroyed.

[MartinNowak]

I know _static_dsos[] needs to be protected with a mutex. Other than that, I believe the design is sound.

This is a bigger issue than it might initially appear. Say that thread A loads a shared library libfoo.so which contains module foo. Because we can only initialize that module for thread A one cannot use that module in thread B.
Using a thread local array for dynamically loaded libraries is necessary to keep track of which library is initialized for which thread.
One consequence of this is that foreach (m; ModuleInfo) might list different modules in different threads.

[WalterBright]

I don't see a fundamental reason why a dll cannot be initialized for all threads.

[WalterBright]

I added a mutex around access to _static_dsos[].

[MartinNowak]

I don't see a fundamental reason why a dll cannot be initialized for all threads.

Because native TLS access can't be redirected, you'd need to alter the GS register to do so.
Furthermore it would require to pause other threads which makes loading a library much more expensive than it ought to be.

[WalterBright]

Suppose Thread 1 loads DLL A. Then Thread 2 loads DLL A. It isn't going to be loaded or initialized again.

Also, there's nothing stopping Thread 3 from calling functions in A.

[WalterBright]

@dawgfoto Ah, I see what you mean. Thread 1 cannot initialize TLS in Thread 2, i.e. cannot call the module tls ctors for Thread 2.

In general it may be a problem we're stuck with. Dynamically loading/unloading DLLs does have some safety issues that I don't see a reasonable way around, such as unloading a DLL while still having function pointers into the DLL.

But in the meantime, this pull does not make things worse, nor does it preclude other designs.

[MartinNowak]

Suppose Thread 1 loads DLL A. Then Thread 2 loads DLL A. It isn't going to be loaded or initialized again.

We can perform initialization for Thread 2 in Runtime.loadLibrary. That would need a mapping from the dlopened handle to the DSO struct and knowing library dependencies. It's something to do after we get a single threaded version to run.

Also, there's nothing stopping Thread 3 from calling functions in A.

That's true and as you said certain programming errors remain possible.
At the same time we shouldn't list uninitialized modules.
A thread local array for dynamically loaded libraries solves this and requires no synchronization.

[andralex]

since there are comments for what functions get used, at best the : notation should be used

[andralex]

@braddr @dawgfoto OK to pull this in the interest of time? If it doesn't lock us out of good designs etc. it would be great. Please advise, thanks.

[WalterBright]

The corresponding documentation:

dlang/dlang.org#375

[MartinNowak]

You don't actually need a mutex because calls to dlopen/dlclose are already globally serialized by the runtime linker, i.e. there is always only one thread calling _d_dso_registry.

[WalterBright]

But there are references to _static_dsos[] outside of _d_dso_registry which do need to be synchronized with the ones inside _d_dso_registry. Otherwise the code may wind up pointing to memory that has been free()'d.

[MartinNowak]

OK to pull this in the interest of time?

I'd rather spend two more days to sort out the remaining issues with this pull.
Also splitting off some changes (i.e. #587, thread_term) would make sense.

[WalterBright]

Also splitting off some changes (i.e. #587, thread_term) would make sense.

I had that as a separate pull already,
https://github.com/D-Programming-Language/druntime/pull/581/files
but merged it in with this one as there seemed no further point of making it separate.

[MartinNowak]

I had that as a separate pull already, #581

Which wasn't immediately merged because it introduced two unrelated behavioral changes.
Those are still debatable within this much bigger pull, so splitting off orthogonal changes will help to focus the discussion and proceed with the merge.

[MartinNowak]

I added #589 to #587.

[MartinNowak]

This call chain comes to early gc_init=>thread_init=>thread_attachThis=>rt.sections_linux.initTLSRanges, because it makes the implicit assumption that all libraries are already loaded. It will miss the TLS ranges from anything but libphobos2.so.

[MartinNowak]

I tried to make an alternative pull for the auto initialization during loading (https://github.com/dawgfoto/druntime/tree/autoInit) and it turns out, that this requires more work because druntime's initialization is pretty messy and contains a lot of implicit assumptions which break when we perform the initialization earlier.
As this is not strictly required to get dynamic loading to work we could defer this.

[MartinNowak]

Got it to work #590.

[WalterBright]

Which wasn't immediately merged because it introduced two unrelated behavioral changes.

They were related because the initialization in rt_init and otherwise was different. They needed to be the same. It would not pass the test suite otherwise.

[MartinNowak]

And dynamic loading #593.

[WalterBright]

This is superceded by #593