msvcrt.dll toolchain

[mirek-fidler]

Now that 10.0.0 is out, is there any chance that msvcrt.dll based toolchain will be added to regular releases?

Obviously, not possible and no need to add ARM tools/libs there. Pure X86/X64.

Ability to link myriads of readily available libs would be awsome....

[mstorsjo]

I'll consider it (but it also bloats the release process a bit) - I do realize, and can definitely acknowledge that there's good reasons for one to want to use that.

It'd allow you to link static libraries that use C, but wouldn't help for anything in C++ anyway.

And regarding ARM, msvcrt.dll does exist there as well (I regularly test the whole toolchain in msvcrt.dll configuration anyway, to make sure it doesn't break, and I do that testing with all four architectures) and works just as on x86, but there's just no reason to use it - compared to on x86.

[mirek-fidler]

Thanks.

…

On Mon, Mar 30, 2020 at 8:45 PM Martin Storsjö ***@***.***> wrote: I'll consider it (but it also bloats the release process a bit) - I do realize, and can definitely acknowledge that there's good reasons for one to want to use that. It'd allow you to link static libraries that use C, but wouldn't help for anything in C++ anyway. And regarding ARM, msvcrt.dll does exist there as well (I regularly test the whole toolchain in msvcrt.dll configuration anyway, to make sure it doesn't break, and I do that testing with all four architectures) and works just as on x86, but there's just no reason to use it - compared to on x86. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARH234YX6LDQARPNHL5IF3RKDSELANCNFSM4LWZ2NKQ> .

[mirek-fidler]

P.S.: If you feel like it is not worth doubling the number of release archive, I believe that x64 version is the one that most people (definitely us) would need. ("llvm-mingw-20200331-x86_64-msvcrt.zip" :)

[mstorsjo]

P.S.: If you feel like it is not worth doubling the number of release archive, I believe that x64 version is the one that most people (definitely us) would need. ("llvm-mingw-20200331-x86_64-msvcrt.zip" :)

Yeah, that, plus the cross compiler version probably. But then I provide the cross compiler both as tarball and docker images... although I don't think the docker images are used (but they're good for myself to dump a 5 GB archive on somebody else's storage for easy access :P), so the cross compiler tarball would probably be enough as well.

[Benau]

is there any docs or require line of change to make a msvcrt.dll toolchain myself?

because last time i tried it failed with something like "codepage" codecvt linking error....

[mstorsjo]

is there any docs or require line of change to make a msvcrt.dll toolchain myself?

If building with the build-all.sh script, add --with-default-msvcrt=msvcrt on the build-mingw-w64.sh line.

because last time i tried it failed with something like "codepage" codecvt linking error....

That sounds unusual - and doesn't sound like something related to switching to using msvcrt. If you try and fail, share logs and I might be able to help.

[Benau]

Ok now works fine, but seems that whenever pthread function is called it crashed... (but for my project supertuxkart i can change all of them to std::thread anyway to avoid the crash)

[Benau]

btw do you think statically link libc++ and libuwind will allow its usage on windows xp? (or < windows 7)

or statically link is not possble at all

[mstorsjo]

btw do you think statically link libc++ and libuwind will allow its usage on windows xp? (or < windows 7)

or statically link is not possble at all

Statically linking them is possible, pass -static when linking, or delete libc++.dll.a and libunwind.dll.a. That doesn't help for running on older versions, as libc++ uses a function that was added in Windows 7.

[mirek-fidler]

Do we know which function is it?

[mstorsjo]

Do we know which function is it?

It's TryAcquireSRWLockExclusive. Both libunwind and libc++ use lots of the new threading/synchronization functions from Vista, but this function, despite belonging to the same family, was introduced later, in Windows 7.

[mirek-fidler]

Well, it would probably be quite a lot of work to replace these, but I am 100% sure you can implement efficiently RW locks using just Win 2000 API. We are using such implementation based on Chris Thomasson code....

[mstorsjo]

Well, it would probably be quite a lot of work to replace these, but I am 100% sure you can implement efficiently RW locks using just Win 2000 API. We are using such implementation based on Chris Thomasson code....

Yes, definitely possible - just quite a lot of code, and probably not upstreamable (I think both llvm itself and the runtimes have win7 as minimum requirement, so they probably don't want to complicate things for compatibility with older versions).

[Benau]

So if my project doesnt use std::shared_mutex, statically link will discard the symbol anyway?

Or some other part of libc++ uses rw lock

[mstorsjo]

So if my project doesnt use std::shared_mutex, statically link will discard the symbol anyway?

Or some other part of libc++ uses rw lock

Unfortunately it's all pretty entangled. I tried building libc++ statically with -ffunction-sections and linking with -Wl,--gc-sections, for a small testcase that prints hello world to std::cout, but I still got AcquireSRWLockExclusive, ReleaseSRWLockExclusive and SleepConditionVariableSRW.

[mirek-fidler]

BTW, what is -static exactly supposed to do?

I have tried compiling with and without and the list of .dlls loaded seems to be exactly the same.

[mstorsjo]

BTW, what is -static exactly supposed to do?

Normally, when then linker is given a -lfoo argument to link against the library foo, it looks for libfoo.dll.a followed by libfoo.a. (It also looks for a few other patterns, but these are the primary ones, for simplicity.) The convention is that libfoo.dll.a is an import library for the library foo.dll, while libfoo.a is a regular static library. But this is purely a convention.

When the linker is given -static (or more precisely, when the compiler driver is given -static, which is passed as -Bstatic to the linker), it only looks for libfoo.a and never for libfoo.dll.a. For libraries where both alternatives are available, this makes sure to link them statically instead of dynamically. For libraries that only are available dynamically (i.e. only libfoo.dll.a, not libfoo.a), it won't find them though.

I have tried compiling with and without and the list of .dlls loaded seems to be exactly the same.

If you just compile a regular C executable that doesn't use anything else than system libraries, there won't be any difference. For C++, it will chooses to link libunwind and libc++ statically instead of dynamically.

The mingw base import libraries are special - they are named e.g. libkernel32.a even though it's an import library for kernel32.dll. If it'd only be named libkernel32.dll.a linking with -static would always fail as there's no statically linked version of this available.

And to be more precise, many of the core mingw import libraries (in particular the various CRT libraries, libmsvcr*.a and libucrt*.a) actually are hybrids - while they serve as an import library against the system DLLs, they also contain a number of statically linked functions, adapting the interface expected by the object files to what the actual DLLs provide.

[mirek-fidler]

For C++, it will chooses to link libunwind and libc++ statically instead of dynamically.

Thanks. In the end it turned out it was my mistake, I was actually having -static in both modes (just in one of them twice... :). After fixing this, I can see references to libunwind and libc++ dlls just as you say.

[mirek-fidler]

I need a hint, if you have a bit of time. While I can now compile in ucrt mode, I still fail to produce working msvcrt.

So after succeding to compile in urcrt, I have changed build-all.sh

.....
./install-wrappers.sh $PREFIX
./build-mingw-w64.sh $PREFIX --with-default-msvcrt=msvcrt
./build-compiler-rt.sh $PREFIX
....

and ran compilation again, to a new target directory. It run for a while, then stopped saying that produced compiler does not work.

Desperately clueless, I tried to delete libssp, llvm-project and mingw-w64 directories with the idea they contain some cached values from the previous build. Rerun, script cloned them again, seemed to have finished. However if I start clang++, it now says it is missing zlib1.dll.

I suspect that the whole thing needs some cleanup to switch to msvcrt mode...?

[mirek-fidler]

A little idea about AcquireSRWLockExclusive, ReleaseSRWLockExclusive and SleepConditionVariableSRW problem. Maybe the easy solution would be to create a little static library that contains these 3 functions and link it with the project? Or even put them into your code...

For starters and single-threaded code, they could be empty. For level 2, it should be possible to actually provide the correct implementation...

[Benau]

Well i can make libc++ to use winpthread,but not libuwind...

not sure if using libgcc is possible....

Even after that it still have some symbol missing in windows xp (like wstombs forgot spelling...)

[mati865]

Maybe the easy solution would be to create a little static library that contains these 3 functions and link it with the project?

That would not help because the kernel doesn't support it.
You could functions that emulate them with other already existing functions but it'd be enormous effort to keep 1:1 compatibility.

not sure if using libgcc is possible....

MSYS2 shipped libcxx has always been using libgcc for unwinding and builtins. AFAIK it still should be the case.
Though it's using win32 threading nowadays since the oldest supported OS is Win7.

[mirek-fidler]

You could functions that emulate them with other already existing functions but it'd be enormous effort to keep 1:1 compatibility.

I think for starters, adding them with empty bodies into application code should at least test the theory that this fixes exe loading problem. And it would probably work in single-threaded code where there is nothing to lock anyway (OTOH, even if the code is single-threaded on the client side, there tend to be library threads and OS threds anyway).

(For next level, emulating with already existing functions should be possible and as for compatibility, I do not think that would be "enormous" problem. Yes, you might have a little bit hard time fitting your data into API structures, but as long as you follow API compatibility shoud not be the issue).

[mstorsjo]

I need a hint, if you have a bit of time. While I can now compile in ucrt mode, I still fail to produce working msvcrt.

Are you compiling using an existing version of llvm-mingw, or using the default msys2 gcc setup, or something else?

So after succeding to compile in urcrt, I have changed build-all.sh

.....
./install-wrappers.sh $PREFIX
./build-mingw-w64.sh $PREFIX --with-default-msvcrt=msvcrt
./build-compiler-rt.sh $PREFIX
....

and ran compilation again, to a new target directory. It run for a while, then stopped saying that produced compiler does not work.

Desperately clueless, I tried to delete libssp, llvm-project and mingw-w64 directories with the idea they contain some cached values from the previous build. Rerun, script cloned them again, seemed to have finished.

Removing those dirs does help for cleaning. I've recently added code to the build scripts that checks for an environment variable named CLEAN - if that's set, it will wipe the existing build dirs (e.g. mingw-w64/mingw-w64-crt/build-x86_64) before building, to ease rebuilding without having to fetch all the code again.

However if I start clang++, it now says it is missing zlib1.dll.

I suspect that the whole thing needs some cleanup to switch to msvcrt mode...?

I don't think that has anything to do with switching. It sounds like cmake, while building clang/llvm, looked for zlib and happened to find it (from your surrounding msys2 install, I'd presume), but then is unable to find the DLL at runtime. If you built it in ucrt mode before I would expect the same to happen there as well though.

[mstorsjo]

I added builds that target msvcrt now in the latest release now; one cross compiler package, and native ones that run on i686 and x86_64.