Emit ModuleInfo stubs for -betterC shared libraries

[rikkimax]

Target: Windows 64bit
Compiler: 1.30-beta1

Scenario:

Shared library built with -betterC (may have static library dependencies), no ModuleInfo.

Full D executable, depends on shared library.

Result:

spew-dem2-base-test-library.obj : error LNK2001: unresolved external symbol _D6sidero4base10allocators2gc12__ModuleInfoZ
spew-dem2-base-test-library.obj : error LNK2001: unresolved external symbol _D6sidero4base10allocators12__ModuleInfoZ
spew-dem2-base-test-library.obj : error LNK2001: unresolved external symbol _D6sidero4base10allocators3api12__ModuleInfoZ
.dub\build\spew-dem2-base-test-library-unittest-windows-x86_64-ldc_v1.30.0-beta1-9ABCC933E5B7E94DDEF0A00077D4D3E6\spew-dem2-base-test-library.exe : fatal error LNK1120: 3 unresolved externals

I managed to work around this by generating ModuleInfo stubs in the shared library via:

module sidero.base.moduleinfostubs;

static foreach (Stub; Stubs) {
    mixin(() {
        import std.format : format;

        //string ret = "export extern(C) __gshared void[] " ~ Stub.mangleName ~ " = cast(void[])[cast(ubyte)";
        string ret = "export extern(C) void " ~ Stub.mangleName ~ "() { asm { naked; ";

        void add(ubyte b) {
            ret ~= "db 0x";

            ubyte temp = b & 0xF;
            if (temp > 9)
                ret ~= 'A' + (temp - 10);
            else
                ret ~= '0' + temp;

            temp = (b >> 4) & 0xF;
            if (temp > 9)
                ret ~= 'A' + (temp - 10);
            else
                ret ~= '0' + temp;

            ret ~= ";";
        }

        add(MIname & 0xFF);
        add((MIname >> 8) & 0xFF);
        add(0);
        add(0);

        add(0);
        add(0);
        add(0);
        add(0);

        foreach (c; Stub.moduleName) {
            add(c);
        }
        add(0);

        return ret ~ "}\n}\n";
        //return ret ~ "];\n";
    }());
}

private:
struct ModuleInfoStub {
    string mangleName;
    string moduleName;
}

enum Stubs = [
        ModuleInfoStub("_D6sidero4base10allocators2gc12__ModuleInfoZ", "sidero.base.allocators.gc"),
        ModuleInfoStub("_D6sidero4base10allocators12__ModuleInfoZ", "sidero.base.allocators"),
        ModuleInfoStub("_D6sidero4base10allocators3api12__ModuleInfoZ", "sidero.base.allocators.api")
    ];

For -betterC code, it would be very nice to be able to tune ModuleInfo emittance (including emitting of stubs).

Note: this does pass my test suite, so yay :D

[rikkimax]

Is it possible that my usage of pragma(crt_constructor) is triggering the dependency on the ModuleInfo?

Specifically[0] causes the sidero.base.allocators.gc to be added by the executable (and not used anywhere else).

[0] https://github.com/Project-Sidero/basic_memory/blob/main/source/sidero/base/allocators/gc_hook.d

[kinke]

Is it possible that my usage of pragma(crt_constructor) is triggering the dependency on the ModuleInfo?

Nope, but some directly or indirectly imported module with a module ctor/dtor. E.g. (IIRC):

• module a has a static this
• module b imports a and is compiled with worseD for some reason => ModuleInfo generation is suppressed (which would contain a ref to the a ModuleInfo)
• module c imports b => its ModuleInfo.importedModules will contain a ref to the b ModuleInfo

This is needed for druntime to establish the module ctor invocation order at startup. These link-time errors are currently the only protection for worseD code depending on modules with module ctors (which of course aren't run without druntime).

What's your 'excuse' for the betterC shared lib? ;)

[rikkimax]

• module a has a static this
• module b imports a and is compiled with worseD for some reason => ModuleInfo generation is suppressed (which would contain a ref to the a ModuleInfo)
• module c imports b => its ModuleInfo.importedModules will contain a ref to the b ModuleInfo

None of these apply.

It's a leaf module, nothing imports it, when commented out (constructor + destructor) the reference to _D6sidero4base10allocators2gc12__ModuleInfoZ disappears.

Context: I'm trying to build up to a full GUI toolkit, and that means things like IDE form editor ext. So I want to be able to mix compilers and compiler versions with its shared libraries in the plugin architecture. With the druntime, it just adds to the mix of things that can go wrong. At least by starting with -betterC, there is a chance that this isn't just a fever dream.

[kinke]

It's a leaf module, nothing imports it, when commented out (constructor + destructor) the reference to _D6sidero4base10allocators2gc12__ModuleInfoZ disappears.

Maybe read again? - This gc_hook leaf module imports sidero.base.allocators.gc. Which most likely imports (directly, or more likely, indirectly) some other module with a module c/dtor, quite likely from druntime. In that case, the sidero.base.allocators.gc_hook ModuleInfo references the (missing) sidero.base.allocators.gc ModuleInfo. Using the LLD linker (-link-internally) should show where the missing symbol is referenced from.

[rikkimax]

It, unfortunately, didn't show it:

Linking...
lld-link: error: duplicate symbol: _D3std8bitmanip__T20nativeToLittleEndianTmZQzFNaNbNiNfxmZG8h
>>> defined at phobos2-ldc.lib(bitmanip.obj)
>>> defined at sidero_base.dll

I described it as a leaf because apart from the crt_constructor/destructor it should never be referenced by anything, certainly not by import in D.

There shouldn't be any module constructors of any kind being in use here. However, this doesn't really help me much, even if this one was fixed (as I narrowed it down, so I know which module is doing it), the other two are much more widely used and almost certainly are going to be hit by the random user.

https://github.com/Project-Sidero/basic_memory/blob/main/source/sidero/base/allocators/gc.d

[rikkimax]

Okay bit more chatting on Discord, and yeah it's looking like it's due to imports requiring it, not constructor/destructor.

So a stub would still need MIimportedModules to be fully working, which my code isn't doing. Which isn't really a problem for this case. But in the medium term working around it won't be viable. Its going to need compiler support i.e.

--emitmoduleinfo=none|stub|full(default)

[rikkimax]

GDC has -fmoduleinfo and -fno-moduleinfo to configure this.