[GC] DLL startup code is out of order; gc proxy not set properly

[dlangBugzillaToGithub]

Walter Bright (@WalterBright) reported this on 2011-01-16T22:08:28Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=5457

Description

DLLs link with gcstub.obj, this is because DLLs share the gc with the caller's gc, instead of having a separate gc that fights the caller's. The general idea is that upon initialization the DLL sets "proxy" to point to the caller's gc, and then all gc calls are routed through the proxy.

First, in our DLL's DllMain(), we call:

        case DLL_PROCESS_ATTACH:
            dll_process_attach(hInstance);

In dll_process_attach(), druntime calls:

    Runtime.initialize()

which calls:

    rt_init(null)

which calls:

   gc_init();
   initStaticDataGC();

which calls:

    gcstub.gc.gc_addRange()

which looks like:

extern (C) void gc_addRange( void* p, size_t sz )
{
    if( proxy is null )
    {
        Range* r = cast(Range*) realloc( ranges,
                                         (nranges+1) * ranges[0].sizeof );
        if( r is null )
            onOutOfMemoryError();
        r[nranges].pos = p;
        r[nranges].len = sz;
        ranges = r;
        ++nranges;
        return;
    }
    return proxy.gc_addRange( p, sz );
}

Note that proxy is null. It is supposed to be initialized by rt_loadLibrary(). But, sadly, it is too late since dll_process_attach() gets called first by LoadLibrary()!

This bug makes D DLLs loaded dynamically from a D program unusable.