LLVM stdenvs with LLD outside of pkgsLLVM

[sternenseemann]

Ever since #122778 llvmPackages_*.stdenv and llvmPackages_*.libcxxStdenv respects stdenv.hostPlatform.linker. This means that it'll be using ld.bfd on Linux and cctools on macOS.

This has a big downside, namely it being virtually impossible to produce C++ binaries using these stdenvs. This seems to be caused by the fact that ld.bfd doesn't really understand the object files clang produces (cctools probably works fine? Haven't tested it on darwin so far). This is quite a shame especially with libcxxStdenv.

Current workarounds for this are either using pkgsLLVM.stdenv (which cross-compiles, however) or using the following incantation: overrideCC llvmPackages.stdenv (llvmPackages.stdenv.cc.override { inherit (llvmPackages) bintools; }).

I've been struggling to find a clear way forward and not opened a PR for this so far, but would really like to fix this in a satisfying way. The question is mostly if we a) want to preserve llvmPackages_*.stdenv using the default linker for the platforms and create a new clang+lld LLVM stdenv with a different name (which one?) or b) revert the meaning of llvmPackages_*.stdenv to the pre 21.05 days and have it use LLD unconditionally.

cc @Ericson2314

[Ericson2314]

It doesn't work with ld.bfd at all? Wow.

I suppose we could just make lldClang again?

[sternenseemann]

It doesn't work with ld.bfd at all? Wow.

It does work to an extent, i. e. you can build re2 with libcxxStdenv. However, things start going wrong when C++ libraries need to link against other C++ libraries with strange linker failures. The impression I've gotten is that the only way this really works is if all involved libraries are compiled using clang and linked using lld.

[oxalica]

I kinda dislike #122778 and want to make llvmPackages.stdenv use LLVM bintools by default.
Using ld.bfd in LLVM stdenv is so surprising and troublesome since we'd expect LLVM stdenv uses LLVM toolchains. And currently in chromium and firefox we actually need to explicitly introduce back ld.lld. I don't think there such a "linker of platform". Linkers are independent from platform. They are details of building process.

[sternenseemann]

I don't think there such a "linker of platform". Linkers are independent from platform. They are details of building process.

Arguably you could also say this about the compiler or the toolchain in general, but it makes sense in the context of a package set generally.

Using ld.bfd in LLVM stdenv is so surprising and troublesome since we'd expect LLVM stdenv uses LLVM toolchains.

I'm also a bit torn on this one, maybe llvmPackages.stdenv should always use LLD. Then again, it does work fine for C and I think it should work fine in general on darwin with cctools?!

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/gcc11stdenv-and-clang/17734/2

[aaronmondal]

Hi, let me resurrect this issue 😃

IMO the llvmPackages_* rules should definitely use lld as linker, as those using LLVM will often rely on features like LLVM bitcode linking and thinLTO, both of which are only really viable with lld. I'd expect that ~99% of users would want to use LLVM with lld, the remaining ~1% being only those developing linkers or explicitly experimenting with linkers for compatibility purposes. LLD has recently introduced a few changes (https://reviews.llvm.org/D135402) that hard-error on some buggy builds, but in those cases the downstream projects should be fixed or the linker explicitly changed only for those specific packages.

Same for bintools. The non-LLVM variants can't handle LLVM bitcode. E.g. if you want to use the clang offloading toolchains you have to use llvm-objcopy because the non-LLVM objcopy will fail.

[ghost]

This is currently one of the major obstacles to pkgsCross.*.chromium.

[reckenrode]

I think it would be surprising on Darwin if something that just needed a newer (or specific) version of clang also got lld in place of ld64. It would be nice if Darwin could use just lld, but it’s unfortunately not there yet.