Building static-linux-musl binary with clang: gcc_eh library not found

[avdv]

Describe the bug

After upgrading nixpkgs to 22.05 from 21.11, trying to build a simple hello world C program, using the pkgsStatic.llvmPackages_11.stdenv or pkgsStatic.llvmPackages.stdenv or pkgsStatic.clangStdenv results in

/nix/store/fnw7g7s8z6wmaank060jpqppl0hiqbal-x86_64-unknown-linux-musl-binutils-2.38/bin/x86_64-unknown-linux-musl-ld: cannot find -lgcc_eh: No such file or directory
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

Steps To Reproduce

Steps to reproduce the behavior:

1. flake.nix:

   {
     inputs.nixpkgs.url = "nixpkgs/nixos-22.05";
   
     outputs = { nixpkgs, ... }:
       let
         system = "x86_64-linux";
         pkgs = import nixpkgs { inherit system; };
         pkgsStatic = pkgs.pkgsStatic;
         stdenvStatic = pkgsStatic.llvmPackages.stdenv;
         # also tried:
         # stdenvStatic = pkgsStatic.clangStdenv;
       in
       {
         packages.${system}.default = stdenvStatic.mkDerivation {
           name = "hello";
           src = ./.;
           unpackPhase = ''
             cat > hello.c <<"EOF"
             #include <stdio.h>
             int main() {
               printf("hello, world\n");
             }
             EOF
           '';
           buildPhase = "$CC -o hello hello.c";
           installPhase = "install -D -t $out/bin hello";
         };
       };
   }

2. nix build .

   $ nix build .
   error: builder for '/nix/store/pgk2n1vcmy4gjkqycbqs1wblmjmwhwpz-hello-x86_64-unknown-linux-musl.drv' failed with exit code 1;
          last 9 log lines:
          > unpacking sources
          > patching sources
          > updateAutotoolsGnuConfigScriptsPhase
          > configuring
          > no configure script, doing nothing
          > building
          > clang-11: warning: argument unused during compilation: '-pie' [-Wunused-command-line-argument]
          > /nix/store/fnw7g7s8z6wmaank060jpqppl0hiqbal-x86_64-unknown-linux-musl-binutils-2.38/bin/x86_64-unknown-linux-musl-ld: cannot find -lgcc_eh: No such file or directory
          > clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
          For full logs, run 'nix log /nix/store/pgk2n1vcmy4gjkqycbqs1wblmjmwhwpz-hello-x86_64-unknown-linux-musl.drv'.

Expected behavior

I am able to build a static executable, based on musl, with clang.

Additional context

It seems like the gcc_eh.a library is indeed missing:

ls /nix/store/d41dvnv1rir038hxx8k8xf57icrfz7l4-x86_64-unknown-linux-musl-stage-final-gcc-11.3.0/lib64/gcc/x86_64-unknown-linux-musl/11.3.0/*.a
   libgcc.a    libgcov.a

Removing the -lgcc_eh option from the linker flags and manually running the resulting command produces a valid executable. (but I suppose this only works since I am not using exceptions ?)

Notify maintainers

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.43, NixOS, 22.05 (Quokka)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - channels(root): `"nixos-21.11.336538.64fc73bd74f, nixos-hardware"`
 - channels(claudio): `"nixos-21.11.334139.1bd4bbd49be"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Note, I am using nix flake to configure my system:

$ nix flake info
Resolved URL:  git+file:///home/claudio/src/nix-config
Locked URL:    git+file:///home/claudio/src/nix-config
Description:   NixOS and Home Manager config
Path:          /nix/store/sy3hlsffnh0mcbx6f597yj9pmcwa70fk-source
Last modified: 2022-06-06 22:44:21
Inputs:
├───flake-compat: github:edolstra/flake-compat/b7547d3eed6f32d06102ead8991ec52ab0a4f1a7
├───flake-utils: github:numtide/flake-utils/3cecb5b042f7f209c56ffd8371b2711a290ec797
├───home-manager: github:nix-community/home-manager/a8d00f5c038cf7ec54e7dac9c57b171c1217f008
│   └───nixpkgs follows input 'nixpkgs'
├───nixos-hardware: github:NixOS/nixos-hardware/816a935bf5aa5f77cb1f03ebfe20ab13b112d0f1
└───nixpkgs: github:NixOS/nixpkgs/d9794b04bffb468b886c553557489977ae5f4c65

[avdv]

I bisected this to this commit: c6dd11c

Revert "gcc: Always pass --enable-shared by default"

This reverts commit https://github.com/NixOS/nixpkgs/commit/8e482321805542e08eaf8819a6ad94ea4ba20370.

Since pkgsStatic.stdenv.cc can only produce static binaries, there's
no reason to include that compilers e.g. libstdc++.so.

@alyssais do you know what might be causing this?

[avdv]

Oh, according to this mailing list entry, this behavior is kind of expected (although it is quite old -- is this still the case?):

In the --enable-shared configuration we have:

libgcc.a - Support routines, not including EH
libgcc_eh.a - EH support routines
libgcc_s.so - Support routines, including EH

In the --disable-shared configuration we have:

libgcc.a - That's all you get, folks - all routines

Which part of the nix cc toolchain adds the flag for libgcc_eh then? Maybe it is just not needed?

\edit: It seems that clang itself is adding these flags for the linker, so it expects libgcc_eh.a to be there when linking statically (e.g. see https://github.com/llvm/llvm-project/blob/26041e17006cf30e9c2d06706fe6bd3fa818e356/clang/test/Driver/linux-ld.c#L171-L177)

I work-around that problem currently by creating an empty libgcc_eh.a library and adding its path to the linker search directories.

[ParetoOptimalDev]

I'm also gettting a problem related to this when trying to cross compile elm after working around my initial issues:

$ nix-build '<nixpkgs>' -A pkgs.pkgsCross.aarch64-multiplatform-musl.pkgsStatic.elmPackages.elm 
# .. snip ..
Node.js configure: Found Python 3.10.5...
Usage: configure [options]

configure: error: no such option: --disable-shared <<<<<<<<------------------------------------------------
error: builder for '/nix/store/f342pp045fpq22lifvzdsg3va6myf22q-nodejs-static-aarch64-unknown-linux-musl-14.19.3.drv' failed with exit code 2;
       last 10 log lines:
       > Updating Autotools / GNU config script to a newer upstream version: ./deps/cares/config.sub
       > Updating Autotools / GNU config script to a newer upstream version: ./deps/cares/config.guess
       > configuring
       > fixing libtool script ./deps/cares/ltmain.sh
       > ./deps/cares/configure
       > configure flags: --prefix=/nix/store/chbmd02dsg1704s4r57md0kza8zi6c6f-nodejs-static-aarch64-unknown-linux-musl-14.19.3 --shared-libuv --shared-libuv-libpath=/nix/store/3crq5kb39ydvbk1fw8zryvxhdznjfyx7-libuv-static-aarch64-unknown-linux-musl-1.44.1/lib --shared-openssl --shared-openssl-libpath=/nix/store/sq9f92bs1y9y2z9rjzs10y1s6ljjhshb-openssl-static-aarch64-unknown-linux-musl-1.1.1p/lib --shared-zlib --shared-zlib-libpath=/nix/store/8wh0syl71qdpzm6syr3sm91lggasapb2-zlib-static-aarch64-unknown-linux-musl-1.2.12/lib --with-intl=system-icu --without-dtrace --cross-compiling --without-intl --without-snapshot --dest-cpu=arm64 --enable-static --disable-shared --disable-shared
       > Node.js configure: Found Python 3.10.5...
       > Usage: configure [options]
       >
       > configure: error: no such option: --disable-shared
       For full logs, run 'nix log /nix/store/f342pp045fpq22lifvzdsg3va6myf22q-nodejs-static-aarch64-unknown-linux-musl-14.19.3.drv'.
error: 1 dependencies of derivation '/nix/store/sdrwvgqbl23d4v3ibqcfim95kvbijp8a-elm-static-aarch64-unknown-linux-musl-0.19.1.drv' failed to build

[avdv]

BTW, there is an upstream bug for LLVM here.

[lilyball]

I just hit this. The nixpkgs clang derivation already patches the source to switch from lgcc_s to lgcc_eh when stdenv.hostPlatform.isMusl is true. I'm not sure why it's looking at hostPlatform instead of targetPlatform though. Anyway, this would suggest that a potential fix (completely untested) is to do a similar patch when !stdenv.targetPlatform.hasStaticLibraries (or maybe !gccForLibs.enableShared, though right now clang doesn't know about that package). It could swap lgcc_eh with lgcc, which I guess would make it try to link against libgcc.a twice, but presumably that's harmless.