Add support for LLD 6.0.0 & switch CI to LDC-LLVM 6.0.0

[kinke]

No description provided.

[kinke]

@thewilsonator: Shall we include SPIR-V in LDC-LLVM 6.0.0? Our ldc-release_60 branch should be ready [incl. compiler-rt as git submodule with automatic build inclusion].

[kinke]

@joakim-noah: It'd be great if you could check our Android compatibility with LLVM 6.0.0 via the source artifact available here (I had to resolve a few conflicts while cherry-picking). Edit: Or just the prebuilt x86_64 artifact from here for the cross-compiler.

[kinke]

Incomplete list of current 6.0.0 regressions:

• Linux x86_64: failing lit-test codegen/exception_stack_trace.d
• Win32: issue LLVM does not accept generated inline assembly with variable+offset #2257 preventing defaultlib compilation
• Win64: failing lit-tests debuginfo/nested_cdb.d and sanitizers/fsanitize_coverage.d

[joakim-noah]

No problem compiling LDC against the pre-built llvm you linked, but it doesn't link the final ldc2 binary. Will investigate further.

[kinke]

but it doesn't link the final ldc2 binary

Same issue on OSX apparently; I guess that's related to the order of the new LLD libs...

[kinke]

The Win32 inline asm issue appears to be triggered by core.cpuid, with enabled optimizations only, so probably has to be considered as LLVM bug.

Failing debuginfo/nested_cdb.d on Win64 (most likely Win32 too) also appears to be a CodeView debuginfo regression (CodeView support is supposed to be greatly improved by v6); once again related to the GEPs we use for nested variables (inner functions only, i.e., where the GEP stems from the bitcast context pointer argument; no problem in the outer function, where it's a GEP from the frame alloca), I can't believe that's bothering us again. :(

[kinke]

Oh @joakim-noah: I can't use the prebuilt LLVMs in my Ubuntu 16.04 VM either. They're built on Ubuntu 14.04 with gcc 4.9, and IIRC there have been breaking ABI changes with newer gcc versions.

[kinke]

At least green on macOS now.

On Win32, I tried replacing the DMD-style inline asm with LLVM asm:

            import ldc.llvmasm;
            __asm(`mov %eax, 2
                   cpuid
                   mov $0, %eax
                   mov $1, %ebx
                   mov $2, %ecx
                   mov $3, %edx`, "=*m,=*m,=*m,=*m,~{eax},~{ebx},~{ecx},~{edx}",
                   &a[0], &a[1], &a[2], &a[3]);

That makes it work with enabled optimizations, but for a debug build, LLVM errors out with inline assembly requires more registers than available. That error vanishes when dropping all clobbers...

[joakim-noah]

OK, I've kicked off a build of the latest llvm-69a834b2.src for Android/ARM, though I had to stop it from generating the build target for the tests with -DLLVM_INCLUDE_TESTS=OFF because of some CMake errors otherwise.

[kinke]

[Ah yes sry I forgot to mention that COMPILER_RT_INCLUDE_TESTS=OFF is required when building LDC-LLVM to get rid of the clang dependency.]

[kinke]

The exception stack trace issue on Linux x86_64 is due to the output of ldc2 -g -link-defaultlib-debug -run ../ldc/tests/codegen/exception_stack_trace.d:

object.Exception@../ldc/tests/codegen/exception_stack_trace.d(6): lala
----------------
runtime.d:779 [0x425dd8]
runtime.d:1035 [0x425718]
dmain2.d:288 [0x40b6d0]
deh.d:24 [0x42a64e]
dwarfeh.d:316 [0x40c1c2]
exception_stack_trace.d:6 [0x4020dd]
exception_stack_trace.d:11 [0x4020e8]
exception_stack_trace.d:18 [0x4020fc]
dmain2.d:508 [0x40bd7c]
dmain2.d:469 [0x40bb59]
dmain2.d:508 [0x40bc7a]
dmain2.d:469 [0x40bb59]
dmain2.d:528 [0x40ba98]
__entrypoint.d:8 [0x402254]

regressing to

object.Exception@../ldc/tests/codegen/exception_stack_trace.d(6): lala
----------------
runtime.d:2734 [0x4187c8]

Only with debug defaultlibs that is; the release libs only show a few less file/line infos compared to LLVM 5.0.1:

object.Exception@../ldc/tests/codegen/exception_stack_trace.d(6): lala
----------------
??:? [0x40e27e]
??:? [0x419a2a]
??:? [0x409efd]
??:? [0x4021ed]
??:? [0x4021f8]
exception_stack_trace.d:688 [0x40220c]
??:? [0x4064bf]
??:? [0x4063d5]
??:? [0x402364]
??:? __libc_start_main [0x2010982f]
??:? [0x402068]

Edit: Erm, the line info in the release stack trace is clearly wrong, that file only has 38 lines. It's correct for LLVM 5.0.1.

[joakim-noah]

Alright, finally got ldc2 to build on linux/x64 and cross-compile for Android/ARM once I built llvm from source. I had to turn LLVM_INCLUDE_TESTS back on and disable COMPILER_RT_INCLUDE_TESTS instead like you said, or the generated llvm-config doesn't work properly, spitting out some weird errors about missing libgtest.a when run with --libs.

All the same stdlib tests pass with your tweaked llvm 6.0 and ldc master, with only that long-standing EH test failing.

[kinke]

LLVM 6.0.0 appears to default to a newer Dwarf debuginfo version (at least for Linux), which the stacktrace generator in druntime doesn't handle yet. This then led to the discovery of a bug in that druntime code, responsible for the previously single strack trace entry when linking against debug druntime.

[kinke]

Should be ready now, except for LDC-LLVM 6.0 finalization (@thewilsonator: SPIR-V?). I promoted the 2 MSVC regressions to separate issues now.

[joakim-noah]

How's the built-in lld working these days? I'm thinking of trying it out for Android and switching to it if it passes all the tests. I could remove the binutils/clang dependency for the Termux Android package too.

[kinke]

Pretty sure it wouldn't cause a lot of issues, but the huge command-line issue remains the same.
With LLD 6, there's a new WebAssembly driver which could prove useful sometime.

[joakim-noah]

Yeah, gathering all those flags is a pain for an arbitrary distro but we could probably do it for more standard setups like Win/Mac or the Android NDK and Termux app. I'll try it out.

[kinke]

[It's working fine on Windows, there's no cmdline hell as on Posix; it's all in environment variables, which are set up by calling an MSVC batch file (still better than having to perform a dummy link with gcc -v and parsing the output). -link-internally works on Windows out of the box.]

[kalev]

Would it be possible to merge these and do a new 1.8.1 release with llvm 6.0 support?

[kinke]

How urgent is the LLVM 6 support? I hope we'll be able to provide a LDC 1.9 beta1 in about 2 weeks and the final in ~4.

[kalev]

I was hoping to be able to switch Fedora 28's LDC to LLVM 6 in time for the Fedora 28 final freeze, which is on April 15. Right now it's stuck with LLVM 4 due to various ppc64el bugs in LLVM 5 (#2356) and now Fedora LLVM people want to remove LLVM 4 from Fedora 28 (https://src.fedoraproject.org/rpms/ldc/pull-request/1).

[kinke]

@kalev: I quickly scanned through the changes. What you could do (for LLVM 6 support) is use -DLDC_WITH_LLD=OFF in the CMake command line; that only prevents (MSVC-targets-exclusive) -link-internally from working. You might need a tiny CMakeLists.txt patch if the C++ compiler doesn't support -Werror=unguarded-availability-new. Then you should only really need (tiny) 18cf419 (default to Dwarf v3 debuginfos) to make LDC 1.8 support LLVM 6.

[kalev]

Ah, awesome, let me try that. Thanks @kinke!

[kinke]

[Oh and you could also achieve the latter by simply adding a new default switch -dwarf-version=3 in the ldc2.conf file.]