libffi stopped working recently on nixos-rebuild

[thedavidmeister]

Issue description

i ran sudo nixos-rebuild switch --upgrade today and got this error:

checking FFI library usage... configure: error: The ffi library (libffi) does not appear to be working.  Perhaps it's missing or you need a more recent version.  Version 3.0.9 or later should work, and you can find a link to it n the list of packages for download at http://www.gnustep.org/resources/sources.html
builder for '/nix/store/pf8g77wi1pb46acrvlnsmshfhqspji84-gnustep-base-1.25.1.drv' failed with exit code 1
cannot build derivation '/nix/store/dck6gghrvv4c645kqkchwqybkypr9ifc-unar-1.10.1.drv': 1 dependencies couldn't be built

Steps to reproduce

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

[thedavidmeister@nixos:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.7, NixOS, 19.03pre161883.e85c1f58680 (Koi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.1.3`
 - channels(root): `"nixos-19.03pre162671.44b02b52ea6, nixpkgs-19.03pre160749.3a393eecafb"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[vielmetti]

looks like unar has changed: https://macpaw.com/blog/macpaw-bought-unarchiver

[matthewbauer]

For future debugging, you should always include config.log when the configure script fails. Reproducing it, this is the error:

configure:10183: clang -o conftest -g -O2  -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/include  -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/lib conftest.c -lbfd -lrt -ldl  -lpthread -lz >&5
In file included from conftest.c:226:
././config/config.proccmd.c:20:36: warning: implicit declaration of function 'getpid' is invalid in C99 [-Wimplicit-function-declaration]
  sprintf(buf, "/proc/%d/cmdline", getpid());
                                   ^
1 warning generated.
configure:10183: $? = 0
configure:10183: ./conftest
./conftest: error while loading shared libraries: libbfd-2.30.so: cannot open shared object file: No such file or directory
configure:10183: $? = 127
configure: program exited with status 127

The breaking commit appears to be: beb063a

not sure what is going on though.

[aij]

@matthewbauer FWIW, I came to the same conclusion via git bisect (starting based on hydra builds https://hydra.nixos.org/build/85622858)

$ git bisect start a27c51 a296b6
$ git bisect run nix-build -A pkgs.gnustep.base
(this took a surprisingly long time... output omited)
$ git bisect log
# bad: [a27c51f71fca2232febdcb44ff1c5ab8521f9e99] premake3: use premake3 -> premake
# good: [a296b6eda746c4e06986cb961a384f6fc99ed449] Merge pull request #51136 from dywedir/rustup
git bisect start 'a27c51' 'a296b6'
# good: [cf9743da48aeb0da81f0932f09ea47bbc14e89e5] remove omake_rc1 from all-packages.nix since it doesn't build
git bisect good cf9743da48aeb0da81f0932f09ea47bbc14e89e5
# bad: [21505290ee534ab5d651389e9facf66b8ae6f40c] json-glib: 1.4.2 → 1.4.4
git bisect bad 21505290ee534ab5d651389e9facf66b8ae6f40c
# bad: [4dda564576a66c499db693348dc6c4175a66775d] gnome3.gnome-backgrounds: 3.28.0 → 3.30.0
git bisect bad 4dda564576a66c499db693348dc6c4175a66775d
# good: [16dabf2d5bc5306f8633b1967a1f9303af4c9963] Merge pull request #50068 from r-ryantm/auto-update/libevdev
git bisect good 16dabf2d5bc5306f8633b1967a1f9303af4c9963
# bad: [071eba26a486e1b86df157a0d5e5d460e740e13a] Merge pull request #51204 from eburimu/fix/cross-f2fs-tools
git bisect bad 071eba26a486e1b86df157a0d5e5d460e740e13a
# bad: [41d887af6f56ce1b9eb1f82ff4a53f9f52bc056d] Merge pull request #51120 from dtzWill/fix/ical-test
git bisect bad 41d887af6f56ce1b9eb1f82ff4a53f9f52bc056d
# good: [edb7146a6fa1bfb1beeae0f78a8bf2f6e07fc92f] Merge pull request #50972 from dtzWill/update/bison-3.2.2
git bisect good edb7146a6fa1bfb1beeae0f78a8bf2f6e07fc92f
# good: [42213e49628a971510f4f585a2029a9bd296679a] Merge pull request #50971 from dtzWill/update/pixman-0.36.0
git bisect good 42213e49628a971510f4f585a2029a9bd296679a
# bad: [beb063a1031c08df77996d97e4f6f9a2da1d911a] binutils: use shared libs
git bisect bad beb063a1031c08df77996d97e4f6f9a2da1d911a
# good: [fb3ed07a2b0120432bf16646469b8aab5a6eec9a] binutils: fix #44936 the huge size regression
git bisect good fb3ed07a2b0120432bf16646469b8aab5a6eec9a
# first bad commit: [beb063a1031c08df77996d97e4f6f9a2da1d911a] binutils: use shared libs

Isn't nixos-unstable supposed to receive updates from master only after the build succeeds on hydra?

[aij]

I don't think it's actually libffi that broke... It's just the way gnustep-base is tesing libffi, namely

configure:10604: clang -o conftest -g -O2  -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/include -I/nix/store/c8nj85khq01x5smzf7yqsy9s4n6m3wnw-libffi-3.2.1-dev/include  -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/lib conftest.c -L/nix/store/la0i822ih43zjkggl9b2v05w4i2l4hs0-libffi-3.2.1/lib/../lib64 -lffi -lbfd -lrt -ldl  -lpthread -lz >&5
configure:10604: $? = 0
configure:10604: ./conftest
./conftest: error while loading shared libraries: libbfd-2.30.so: cannot open shared object file: No such file or directory
configure:10604: $? = 127

In particular, note -lbfd on the first line without a corresponding -L.

Rather strangely, earlier tests specifically for bfd seem to have succeeded. Eg:

configure:8798: checking for bfd_openr in -lbfd
configure:8823: clang -o conftest -g -O2  -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/include -I/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/include  -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/local/lib -L/nix/store/yj26ld5gwh5s0av0karvrx5q8cbrz4j4-gnustep-make-2.7.0/lib conftest.c -lbfd  -lrt -ldl  -lpthread >&5
configure:8823: $? = 0

I'm not sure how...

[matthewbauer]

Nice work! I suspect that libbfd is hardcoded into clang somewhere. For whatever reason it does not pick up the right location though.

Isn't nixos-unstable supposed to receive updates from master only after the build succeeds on hydra?

This is true for certain things that are blocking jobs (or dependencies of those jobs). They are listed here:

https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents

but right now we don't have gnustep or unar in any of these.

[aij]

Oh ok, I suppose blocking only on "important" things makes sense.

Also, thinking about it a little more, I'm not entirely sure libbfd needs to be in propagatedBuildInputs (vs nativeBuildInputs). Running ldd on an older unar, libbfd doesn't show up anywhere. That could be either because it was being statically linked, or because it was only used at build time. (I'm somewhat familiar with libbdf from work I did years ago, but not at all familiar with libffi.)

[thedavidmeister]

hi, i'm still getting this error, what can i do to debug/fix this?

[sboosali]

fwiw, I have this error when building ghc via musl (i.e. pkgsCross.musl64.haskell.packages.ghc863).