Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: Make LDC package fully self-sufficient (get rid of MSVC dependency) #2886

Merged
merged 5 commits into from
Oct 30, 2018

Conversation

kinke
Copy link
Member

@kinke kinke commented Oct 27, 2018

When experimenting with the MinGW-w64 based libs, there was a problem (well, one of many!) - an undefined reference to function _(_)chkstk, emitted by LLVM for stack probing (e.g., after an alloca) and not provided by MinGW-w64.

Initially, I used the symbol exported by ntdll.dll; normal user apps aren't supposed to depend on that system DLL though, and it also led to duplicate symbols for 32-bit code.

After some digging, I found that LLVM's builtins compiler-rt library contains 2 implementations (1 MS-compatible, 1 for MinGW/Cygwin, both with different names than the MS one). Our library didn't contain it though, so I digged deeper.

I found the culprit in the builtins CMake script; when using an MSVC toolchain, the builtins lib on Windows only contains very few C files and none of the various ASM ones, because they use AT&T syntax.

So I decided to try building LLVM & compiler-rt with clang instead. That works fine, but I had to fix the CMake script, as the ASM files are excluded for MS-compatible toolchains, incl. clang. So I forked
compiler-rt to fix this and some more annoyances (CMake fixes only, no code changes).

When compiling LLVM with clang-cl (cl.exe-compatible wrapper), the LLVM C++ compile flags include some clang-specific ones, so building LDC with MSVC didn't work. While that can be fixed at some point (stripping the clang-specific ones), I decided to build LDC (and the C/C++/ASM files in our default libs) with clang too. That required some adaptations, as it started out with > 5k warnings (and crashes in druntime). ;)

So this commit switches to clang for compiling C/C++/ASM, but still uses Microsoft's linker.


Some of the required adaptations are already in master (7b7e0e9, c0d269d, ldc-developers/druntime@b2c58c7, ldc-developers/druntime@9250b1c, ldc-developers/druntime@4afc054). I appended them to the merge-2.083 PR in order to make sure everything still works with the regular MSVC toolchain.

The compiler-rt changes will be pushed as soon as this works.

When experimenting with the MinGW-w64 based libs, there was a problem -
an undefined reference to function `_(_)chkstk`, emitted by LLVM for
stack probing (e.g., after an `alloca`) and not provided by MinGW-w64.

Initially, I used the symbol exported by ntdll.dll; normal user apps
aren't supposed to depend on that system DLL though, and it also led to
duplicate symbols for 32-bit code.

After some digging, I found that LLVM's builtins compiler-rt library
contains 2 implementations (1 MS-compatible, 1 for MinGW/Cygwin, both
with different names than the MS one). Our library didn't contain it
though, so I digged deeper.

I found the culprit in the builtins CMake script; when using an MSVC
toolchain, the builtins lib on Windows only contains very few C files
and none of the various ASM ones, because they use AT&T syntax.

So I decided to try building LLVM & compiler-rt with clang instead. That
works fine, but I had to fix the CMake script, as the ASM files are
excluded for MS-*compatible* toolchains, incl. clang. So I forked
compiler-rt to fix this and some more annoyances.

When compiling LLVM with clang-cl, the LLVM C++ compile flags include
some clang-specific ones, so building LDC with MSVC didn't work. While
that can be fixed at some point (stripping the clang-specific ones), I
decided to build LDC (and the C/ASM files in our default libs) with
clang too. That required some adaptations, as it started out with > 5k
warnings. ;)

So this commit switches to clang for compiling C/C++/ASM, but still uses
Microsoft's linker.
@kinke
Copy link
Member Author

kinke commented Oct 28, 2018

AppVeyor retriggered due to new prebuilt Win32 LLVM package (this time with code changes for compiler-rt on Win32, see https://github.com/ldc-developers/llvm/commits/mingw-w64).

The MSVC detection logic still needs to be adapted in order to fall back to the MinGW-w64 libs (and those libs be bundled in the AppVeyor script); a std.stdio hello-world & runnable/eh[2].d now work on both Win32/64 and with both LLD and MS linker.

@kinke kinke changed the title AppVeyor: Build with clang (incl. LLVM) Windows: Make LDC package fully self-sufficient (get rid of MSVC dependency) Oct 28, 2018
@kinke
Copy link
Member Author

kinke commented Oct 28, 2018

Alright, this should be finally complete now. ldc2 -run hello.d working fine in a naked cmd.exe.

@thewilsonator
Copy link
Contributor

Nice!

@kinke
Copy link
Member Author

kinke commented Oct 29, 2018

Oh right, I forgot that the lit-tests discard most environment variables, so that it's trying to use the MinGW libs, but they are copied after installation. So that's the reason these tests were so much slower on Windows (auto-detecting MSVC and setting up the env variables for each linking operation, ~1 second each - taking > 1 minute in total, while Linux is done in ~12 secs IIRC), I was always wondering about that...

@kinke
Copy link
Member Author

kinke commented Oct 29, 2018

Still not blazingly fast, but the lit-tests runtime went down from ~105 secs to ~45 secs for AppVeyor, i.e., saving one minute and decreasing the chance of timeouts. The clang-builds seem to be somewhat faster too, about ~3 minutes for the full job; I expected a speed up, but hoped it'd be greater. ;)

Using a package generated via the tool and C/ASM sources in the new
https://github.com/ldc-developers/mingw-w64-libs repo, based on
MinGW-w64 v6.0.0.
In order not to fallback to the MinGW libs, which are most likely not
available at testing time. Use the MSVC toolchain used for building
instead, by forwarding the minimum set of required variables.

This also greatly reduces the overall runtime of these tests, as the VS
installation isn't auto-detected (incl. env variables setup) for every
single linking operation.
@kinke kinke merged commit c6053a4 into ldc-developers:master Oct 30, 2018
@kinke kinke deleted the mingw-w64 branch October 30, 2018 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants