Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgsMusl.libffi fails in test suite #79863

Closed
basvandijk opened this issue Feb 11, 2020 · 12 comments · Fixed by #80006
Closed

pkgsMusl.libffi fails in test suite #79863

basvandijk opened this issue Feb 11, 2020 · 12 comments · Fixed by #80006

Comments

@basvandijk
Copy link
Member

basvandijk commented Feb 11, 2020

Describe the bug

On the latest nixpkgs master (8cc8b3b):

$ nix-build -A pkgsMusl.libffi
these derivations will be built:
  /nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv
building '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv'...
...
running tests
check flags: SHELL=/nix/store/sydxyv59z4dka8h22if5v5p11nl1sf4w-bash-4.4-p23/bin/bash VERBOSE=y check
MAKE x86_64-pc-linux-musl : 0 * check
make[1]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl'
Making check in include
make[2]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/include'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/include'
Making check in testsuite
make[2]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make  check-DEJAGNU
make[3]: Entering directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
Making a new site.exp file ...
srcdir='../../testsuite'; export srcdir; \
EXPECT=expect; export EXPECT; \
if /nix/store/sydxyv59z4dka8h22if5v5p11nl1sf4w-bash-4.4-p23/bin/bash -c "runtest --version" > /dev/null 2>&1; then \
  exit_status=0; l='libffi'; for tool in $l; do \
    if runtest --tool $tool --srcdir $srcdir  ; \
    then :; else exit_status=1; fi; \
  done; \
else echo "WARNING: could not find 'runtest'" 1>&2; :;\
fi; \
exit $exit_status
WARNING: Couldn't find the global config file.
Using ../../testsuite/lib/libffi.exp as tool init file.
Test run by nixbld on Tue Feb 11 20:53:11 2020
Target is .runtest-wrapped
Host   is x86_64-pc-linux-musl

                === libffi tests ===

Schedule of variations:
    unix

Running target unix
Using /nix/store/ca489ppzi5p23dv9bk5l3dmcjsd4sasr-dejagnu-1.6.2/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /nix/store/ca489ppzi5p23dv9bk5l3dmcjsd4sasr-dejagnu-1.6.2/share/dejagnu/config/unix.exp as generic interface file for target.
Using ../../testsuite/config/default.exp as tool-and-target-specific interface file.
WARNING: Assuming target board is the local machine (which is probably wrong).
You may need to set your DEJAGNU environment variable.
Running ../../testsuite/libffi.bhaible/bhaible.exp ...
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=1 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O0 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=1 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O2 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=2 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O0 (test for excess errors)
FAIL: libffi.bhaible/test-call.c -W -Wall -Wno-psabi -DDGTEST=2 -Wno-unused-variable -Wno-unused-parameter -Wno-unused-but-set-variable -Wno-uninitialized -O2 (test for excess errors)
...
Running ../../testsuite/libffi.call/call.exp ...
FAIL: libffi.call/align_mixed.c -W -Wall -Wno-psabi -O0 (test for excess errors)
FAIL: libffi.call/align_mixed.c -W -Wall -Wno-psabi -O2 (test for excess errors)
FAIL: libffi.call/align_stdcall.c -W -Wall -Wno-psabi -O0 (test for excess errors)
...
Running ../../testsuite/libffi.closures/closure.exp ...
Running ../../testsuite/libffi.complex/complex.exp ...
Running ../../testsuite/libffi.go/go.exp ...

                === libffi Summary ===

# of expected passes            280
# of unexpected failures        264
# of unsupported tests          206
make[3]: *** [Makefile:449: check-DEJAGNU] Error 1
make[3]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make[2]: *** [Makefile:526: check-am] Error 2
make[2]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl/testsuite'
make[1]: *** [Makefile:1347: check-recursive] Error 1
make[1]: Leaving directory '/build/libffi-3.3/x86_64-pc-linux-musl'
make: *** [Makefile:2949: check] Error 2
builder for '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv' failed with exit code 2
error: build of '/nix/store/pwcgdnmpym76ljmllzydsv7d1kvzdziw-libffi-3.3.drv' failed

Metadata

 nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.98, NixOS, 19.09pre-git (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.2`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: libffi
@basvandijk
Copy link
Member Author

@nh2 seen this before by any chance?

@basvandijk
Copy link
Member Author

Note that pkgsMusl.libffi builds successfully with 82875a2.

@basvandijk
Copy link
Member Author

git bisect comes up with 2476d6d as the first bad commit:

$ git bisect log
git bisect start
# bad: [8cc8b3b235368fad651229cfe9037cd236982f70] Merge pull request #73777 from doronbehar/package-brotab
git bisect bad 8cc8b3b235368fad651229cfe9037cd236982f70
# good: [82875a20ba444110396f95537b18247898e40e22] wasm-bindgen-cli: 0.2.57 -> 0.2.58
git bisect good 82875a20ba444110396f95537b18247898e40e22
# good: [cd27940df06d39353bd47c2cea096e9c8c026591] Merge pull request #78188 from andir/buildRustCrate-support-rlib
git bisect good cd27940df06d39353bd47c2cea096e9c8c026591
# good: [f4f39fc8248ea89f50c08259154f48a7ceda9216] Merge pull request #79309 from jtojnar/hugsie-pkgs
git bisect good f4f39fc8248ea89f50c08259154f48a7ceda9216
# bad: [8847d57763ec4bdb71264fc0f19b24901a345705] Merge staging-next into staging
git bisect bad 8847d57763ec4bdb71264fc0f19b24901a345705
# good: [8396961c9cc67d0a12c0bcf475de77d9a849fd7b] Merge pull request #79300 from jtojnar/default-wm-fix
git bisect good 8396961c9cc67d0a12c0bcf475de77d9a849fd7b
# good: [ed095a6965af671e8478a56446d77f4a4591521c] Merge pull request #74666 from r-ryantm/auto-update/sysstat
git bisect good ed095a6965af671e8478a56446d77f4a4591521c
# bad: [feaf32c35337562c26b48f2f8f6a7f16e5386eca] gmp: 6.1.2 -> 6.2.0
git bisect bad feaf32c35337562c26b48f2f8f6a7f16e5386eca
# bad: [c24fc44e1f1cd7fb4b4f25d174f9284b0d85a154] jfbview: fix build w/glibc-2.30
git bisect bad c24fc44e1f1cd7fb4b4f25d174f9284b0d85a154
# bad: [8e7bbb7d4f70957f1bb7ed049cb038236be58bf7] postfix: fix build w/glibc 2.30
git bisect bad 8e7bbb7d4f70957f1bb7ed049cb038236be58bf7
# good: [c17058226ae769e3666949564904fa71beb5ded6] libunistring: disable tests
git bisect good c17058226ae769e3666949564904fa71beb5ded6
# bad: [2d36eac7801824d5b6d7cc43958b9733e5901109] stdenv: rebuild aarch64 bootstrap files with gcc8
git bisect bad 2d36eac7801824d5b6d7cc43958b9733e5901109
# good: [01ce36f7883cd8fbbeeb3d023222ef031228f6bd] bzip2: fix aarch64 build
git bisect good 01ce36f7883cd8fbbeeb3d023222ef031228f6bd
# bad: [2476d6decabf1be63151dd94be15e87fad52fb0b] expect: fix aarch64 build
git bisect bad 2476d6decabf1be63151dd94be15e87fad52fb0b
# first bad commit: [2476d6decabf1be63151dd94be15e87fad52fb0b] expect: fix aarch64 build

@basvandijk
Copy link
Member Author

@Ma27 any idea why 2476d6d breaks pkgsMusl.libffi?

@basvandijk
Copy link
Member Author

basvandijk commented Feb 12, 2020

Note that removing the autoreconfHook from expect's nativeBuildInputs prevents the pkgsMusl.libffi test failure.

@Ma27
Copy link
Member

Ma27 commented Feb 12, 2020

Oof, that sounds rather nasty :(
Currently building the packages on the community builder, will try to investigate what's going on there today :)

@basvandijk
Copy link
Member Author

I compared two builds of expect. One with autoreconfHook and one without.

When running without autoreconfHook I see the following during configuring:

checking stty to use... /nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty

When running with autoreconfHook I see it setting the wrong path to stty:

checking stty to use... /bin/stty

Also when I compare the configure scripts between the builds I see the following diff:

  • nix-build-expect-5.45.4.drv-0: without autoreconfHook
  • nix-build-expect-5.45.4.drv-1: with autoreconfHook
--- /tmp/nix-build-expect-5.45.4.drv-0/expect5.45.4/configure	2020-02-12 14:58:13.913396295 +0000
+++ /tmp/nix-build-expect-5.45.4.drv-1/expect5.45.4/configure	2020-02-12 14:58:52.129964483 +0000
@@ -730,7 +730,6 @@
 docdir
 oldincludedir
 includedir
-runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -811,7 +810,6 @@
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
-runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
@@ -1064,15 +1062,6 @@
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;

-  -runstatedir | --runstatedir | --runstatedi | --runstated \
-  | --runstate | --runstat | --runsta | --runst | --runs \
-  | --run | --ru | --r)
-    ac_prev=runstatedir ;;
-  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
-  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
-  | --run=* | --ru=* | --r=*)
-    runstatedir=$ac_optarg ;;
-
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1210,7 +1199,7 @@
 for ac_var in	exec_prefix prefix bindir sbindir libexecdir datarootdir \
 		datadir sysconfdir sharedstatedir localstatedir includedir \
 		oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-		libdir localedir mandir runstatedir
+		libdir localedir mandir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1363,7 +1352,6 @@
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
-  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -8639,10 +8627,10 @@

 { $as_echo "$as_me:${as_lineno-$LINENO}: checking stty to use" >&5
 $as_echo_n "checking stty to use... " >&6; }
-if test -r /usr/local/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty ; then
-  STTY_BIN=/usr/local/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty
+if test -r /usr/local/bin/stty ; then
+  STTY_BIN=/usr/local/bin/stty
 else
-  STTY_BIN=/nix/store/ignm17b032p59sfx1mbixn4kfl7g5aa3-coreutils-8.31/bin/stty
+  STTY_BIN=/bin/stty
 fi
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STTY_BIN" >&5
 $as_echo "$STTY_BIN" >&6; }

Why would autoreconfHook cause an invalid path to stty?

basvandijk added a commit to dfinity-lab/nixpkgs that referenced this issue Feb 12, 2020
@Ma27
Copy link
Member

Ma27 commented Feb 12, 2020

Why would autoreconfHook cause an invalid path to stty?

No idea tbh.... But the build was pretty weird before - I still don't exactly know why the addition was needed.

Does expect currently build fine on aarch64 without the autoreconfHook btw?

@basvandijk
Copy link
Member Author

I'll make a PR out of dfinity-lab@a4e23f3 and ask ofborg to build expect on aarch64.

basvandijk added a commit to dfinity/motoko that referenced this issue Feb 13, 2020
This upgrades `nixpkgs` to DFINITY's fork of nixpkgs which tracks the
`master` branch. There're only two patches on top of `master`:

```
commit a4e23f32f8d7006e217125a9123b53597b9c0c24 (HEAD -> dfinity-motoko, dfinity-lab/dfinity-motoko)
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:   Wed Feb 12 16:08:05 2020 +0100

    expect: remove autoreconfHook since that causes a wrong path to stty

    See: NixOS/nixpkgs#79863

commit f359aea057db81c51e55d7250283a0b2201bd6b6
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:	Thu Feb 6 19:14:09 2020 +0100

    libuv: 1.34.1 -> 1.34.2

    (cherry picked from commit 21ad5c123b8e76ab241b89acbc8db24925244d63)
```

We need this patch in order to build `motoko` without using the
upstream `cache.nixos.org`. It would be good not to use an external
cache to decrease the chance an attacker can infect the cache with a
trojan and to ensure DFINITY can always build their own code without
depending on an external cache.
basvandijk added a commit to dfinity/motoko that referenced this issue Feb 13, 2020
This upgrades `nixpkgs` to DFINITY's fork of nixpkgs which tracks the
`master` branch. There're only two patches on top of `master`:

```
commit a4e23f32f8d7006e217125a9123b53597b9c0c24 (HEAD -> dfinity-motoko, dfinity-lab/dfinity-motoko)
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:   Wed Feb 12 16:08:05 2020 +0100

    expect: remove autoreconfHook since that causes a wrong path to stty

    See: NixOS/nixpkgs#79863

commit f359aea057db81c51e55d7250283a0b2201bd6b6
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:	Thu Feb 6 19:14:09 2020 +0100

    libuv: 1.34.1 -> 1.34.2

    (cherry picked from commit 21ad5c123b8e76ab241b89acbc8db24925244d63)
```

We need this patch in order to build `motoko` without using the
upstream `cache.nixos.org`. It would be good not to use an external
cache to decrease the chance an attacker can infect the cache with a
trojan and to ensure DFINITY can always build their own code without
depending on an external cache.
mergify bot pushed a commit to dfinity/motoko that referenced this issue Feb 13, 2020
This upgrades `nixpkgs` to DFINITY's fork of nixpkgs which tracks the
`master` branch. There're only two patches on top of `master`:

```
commit a4e23f32f8d7006e217125a9123b53597b9c0c24 (HEAD -> dfinity-motoko, dfinity-lab/dfinity-motoko)
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:   Wed Feb 12 16:08:05 2020 +0100

    expect: remove autoreconfHook since that causes a wrong path to stty

    See: NixOS/nixpkgs#79863

commit f359aea057db81c51e55d7250283a0b2201bd6b6
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:	Thu Feb 6 19:14:09 2020 +0100

    libuv: 1.34.1 -> 1.34.2

    (cherry picked from commit 21ad5c123b8e76ab241b89acbc8db24925244d63)
```

We need this patch in order to build `motoko` without using the
upstream `cache.nixos.org`. It would be good not to use an external
cache to decrease the chance an attacker can infect the cache with a
trojan and to ensure DFINITY can always build their own code without
depending on an external cache.
basvandijk added a commit to basvandijk/nixpkgs that referenced this issue Feb 13, 2020
@nh2
Copy link
Contributor

nh2 commented Feb 15, 2020

@nh2 seen this before by any chance?

Indeed, my CI noticed this failure, the first failing build is https://buildkite.com/nh2/static-haskell-nix/builds/287#52588b27-84ce-4cd7-97ee-4496fcaa2e37

nh2 added a commit to nh2/static-haskell-nix that referenced this issue Feb 15, 2020
@nh2
Copy link
Contributor

nh2 commented Feb 15, 2020

I'm building static-haskell-nix on top of your patch here:

https://buildkite.com/nh2/static-haskell-nix/builds/291#8eed5f13-bcf1-48c8-b084-600150bad27b

Also noticed that another thing broke though:

  • In that build, apparently I can no longer previous.fetchurl.override, it says value is a function while a set was expected.

nh2 added a commit to nh2/static-haskell-nix that referenced this issue Feb 15, 2020
@nh2
Copy link
Contributor

nh2 commented Feb 16, 2020

* In that build, apparently I can no longer `previous.fetchurl.override`, it says `value is a function while a set was expected`.

My bad, I ran the wrong commit.

This is the real build: https://buildkite.com/nh2/static-haskell-nix/builds/294

#80006 does seem to fix the problem; my working set already succeeded (other package sets still building) all green.

dtzWill pushed a commit to dtzWill/nixpkgs that referenced this issue Feb 17, 2020
grahamc pushed a commit to grahamc/nixpkgs that referenced this issue Feb 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants