fix Issue 23535 - extend pragma(crt_constructor) with semantics that …

[WalterBright]

…static constructors have

1. allow crt_constructors to modify immutable variables
2. make crt_constructors @System because of (1)
3. make crt_constructors and crt_destructors use C linkage, because the C runtime expects that

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#14669"

[WalterBright]

Fixing this is necessary to make progress on #14665

[kinke]

extern(C) is problematic wrt. name clashes due to no mangling. It's the main reason LDC hasn't rejected extern(D) CRT c/dtors - they are ABI-compatible anyway.

[WalterBright]

I know, but them being ABI-compatible is happenstance. I figure that to be pedantically correct, they should use C linkage. I can add a "best practice" note in the documentation that adding a module name prefix to it would be good.

[kinke]

It's happenstance for Win32 only, otherwise the D ABI spec defines it as C-compatible. IMO, requiring the user to unique the names is clearly a PITA and in no way justifies this pedantic-ness about the linkage.

[kinke]

Some evidence:

dmd/druntime/src/core/memory.d

Lines 186 to 206 in ad31ad8

	// The reason for this elaborated way of declaring a function is:
	//
	// * `pragma(crt_constructor)` is used to declare a constructor that is called by
	// the C runtime, before C main. This allows the `pageSize` value to be used
	// during initialization of the D runtime. This also avoids any issues with
	// static module constructors and circular references.
	//
	// * `pragma(mangle)` is used because `pragma(crt_constructor)` requires a
	// function with C linkage. To avoid any name conflict with other C symbols,
	// standard D mangling is used.
	//
	// * The extra function declaration, without the body, is to be able to get the
	// D mangling of the function without the need to hardcode the value.
	//
	// * The extern function declaration also has the side effect of making it
	// impossible to manually call the function with standard syntax. This is to
	// make it more difficult to call the function again, manually.
	private void initialize();
	pragma(crt_constructor)
	pragma(mangle, `_D` ~ initialize.mangleof)
	private extern (C) void initialize() @system

[Geod24]

A technique I usually use:

alias SigHandlerT = extern(C) void function (int sig) nothrow;

/// Returns a signal handler
/// This routine is there solely to ensure the function has a mangled name,
/// and doesn't accidentally conflict with other code.
private SigHandlerT getSignalHandler () @safe pure nothrow @nogc
{
    extern(C) void signalHandler (int signal) nothrow
    { }
    return &signalHandler;
}

[WalterBright]

I wouldn't expect this to get D mangling, but it's what this PR would do currently.

There are other existing cases where this happens, you can see it in the function where I made this change. If crt_constructors are used as intended, one would never see the name mangling.

and also clearly visible from Walter's pragma(mangle) stuff to keep it backwards-compatible.

What was done in the gc code in this PR was the original code was clearly the wrong way to do things, it just happened to work. To minimize the diffs in this PR, I just made it work. The better solution is to put the constructors in their own module, so the name monkey business would not be necessary, it would just work.

@kinke it did not break existing code with the C mangling it did before, but you asked to change it. What else can I do? I cannot fix the ModuleInfo bugs many are complaining about without changing something in the way crt_constructors work. What do you suggest?

[kinke]

but you asked to change it.

Nope. I'm just advocating for leaving the linkage alone - as LDC did from the start, or for a very long time at least. No ModuleInfo bugs require an extern(C) linkage for CRT c/dtors.

[kinke]

I.e., keeping up https://dlang.org/changelog/2.101.0.html#dmd.crt_constructor_signature.

[WalterBright]

Leaving it alone means it has D linkage, which is assumed to be the same as C linkage. It is not, even if it is the same for some platforms. It is a hack to rely on them being the same.

Thanks for the reference. This is recent, and documenting what is going on is not fixing it. C linkage is required because the C runtime library expects it. We cannot define it away.

No ModuleInfo bugs require an extern(C) linkage for CRT c/dtors.

Right. It's the C runtime library that requires it. Fixing the ModuleInfo problem requires fixing the problems with crt_constructors so they are specified properly.

[rikkimax]

Fixing the ModuleInfo problem requires fixing the problems with crt_constructors so they are specified properly.

Oh, I see where you are going with this, it's for the patch function!

EDIT: on second thought, maybe I'm thinking too far ahead oops lol

[WalterBright]

See also #13751

[PetarKirov]

Since D's static constructors already have the desirable semantics, can we make them use the same ABI implementation as CRT constructors, if they have the extern (C) attribute?

[kinke]

They are and always will be ABI-compatible (extern(D) at least), and Walter knows that. - The semantics are quite different though - CRT constructors run before C main and druntime init, in an indeterministic order, while module constructors are run during druntime init, and in an order corresponding to the imports graph, incl. cycles check.

[WalterBright]

They are and always will be ABI-compatible (extern(D) at least), and Walter knows that.

C linkage is defined by the associated C compiler. D linkage is defined by us. C runtime library constructors are defined by C. Not D.

[kinke]

I know all that, and I know that the D ABI spec is C-compatible (and DMD breaks it wrt. params order, but doesn't matter here for void () signatures). And for dying 32-bit Windows, the 'custom' D ABI is compatible here too.

[WalterBright]

The point of having a D ABI is so we are not dependent on the vagaries of the C one, and we can customize it as we like. Having consistency in the specification is also important. Special corners make the language harder to document and understand. We can do special cases, but they need a really strong reason, like there being no other way. That isn't the case here.

[kinke]

I'll just be repeating myself, so one last emphasis: you're being pedantic here and trying to solve an issue that will never ever be a problem (and a trivial test can make sure it doesn't break). While namespace pollution is a very real issue and requires ugly workarounds such as the druntime one I linked above (incl. a long explanatory comment). At work, we have a project consisting of more than 150 D libs, and numerous C++ and C libs. Symbol collisions is a very real threat.

I'll definitely keep accepting extern(D) for LDC. One such CRT ctor is used for DSO registration and so well tested.

[WalterBright]

you're being pedantic here

Yes, I am. D shouldn't be a collection of hacks if we can help it.

Symbol collisions is a very real threat.

And this PR solves it.

[dkorpel]

Yes, I am. D shouldn't be a collection of hacks if we can help it.

An extern(C) function not getting extern(C) mangling feels like a hack though.

[dkorpel]

There's no test for marking a crt constructor @safe or calling it from @safe code.

[dkorpel]

Otherwise, I'm fine with going through with the extern(C) linkage + extern(D) mangling scheme, because it's better than stalling. Is that okay @kinke ?

[kinke]

Well, I think the current proposal goes against the principle of least surprise - the linkage being overwritten to extern(C), and the mangling being special. Leaving the linkage alone would solve that, prevent having to complicate the spec for these special cases, and be backwards-compatible (with v2.101 for extern(D) CRT ctors, and all previous versions for extern(C)). So to me, that's clearly preferable and no hack.

The end result is the same though (no real symbol conflict potential), and that's what I care most about, so I could reluctantly live with the current proposal. ;)

[ibuclaw]

I'll just be repeating myself, so one last emphasis: you're being pedantic here and trying to solve an issue that will never ever be a problem (and a trivial test can make sure it doesn't break). While namespace pollution is a very real issue and requires ugly workarounds such as the druntime one I linked above (incl. a long explanatory comment). At work, we have a project consisting of more than 150 D libs, and numerous C++ and C libs. Symbol collisions is a very real threat.

I'll definitely keep accepting extern(D) for LDC. One such CRT ctor is used for DSO registration and so well tested.

Agreed, forcing extern(C) linkage makes no sense. It's a void function without any parameters (and indeed, called without any parameters too).

[WalterBright]

forcing extern(C) linkage makes no sense

Except that the C calls to it expect C linkage and this change doesn't hurt anything.

[ibuclaw]

Except that the C calls to it expect C linkage and this change doesn't hurt anything.

Both extern(C) and extern(D) get external linkage, even then I don't see it mattering so long as its address is callable through a pointer (so even functions with internal linkage would work with pragma(crt_constructor)).

[WalterBright]

Expecting a C ABI but being provided a D ABI is just poor design, even if it happens to work. And there's no good reason to do this. The language specification shouldn't be a barrage of funky corner cases and special exceptions if we can avoid it. We can avoid it in this case.

[rikkimax]

After reading with a clear head, I think there are two ways forward here:

1. pragma(crt_constructor) does whatever the user tells it to do. If that just so happens to be erroneous oh well. This allows for taking a pointer to the function working as expected and it should mostly "just work".
2. Module constructors if given extern(C) will then be turned into crt_constructors with D mangling and C calling convention.

Will this strategy satisfy everyone? @WalterBright @kinke @ibuclaw

[WalterBright]

I removed the hackish nonsense in the gc constructors so that the mangling of the crt_constructor function becomes irrelevant.