Test suite fails on `aarch64`

[sternenseemann]

https://hydra.nixos.org/build/142576371/nixlog/1:

setupCompilerEnvironmentPhase
Build with /nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4.
unpacking sources
unpacking source archive /nix/store/3v18axdzzvy9gwffd5rmgg54h6s6r45r-alpaca-netcode-0.1.0.0.tar.gz
source root is alpaca-netcode-0.1.0.0
setting SOURCE_DATE_EPOCH to timestamp 1000000000 of file alpaca-netcode-0.1.0.0/test/Test.hs
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/build/setup-package.conf.d -j16 +RTS -A64M -RTS -threaded -rtsopts
[1 of 1] Compiling Main             ( Setup.hs, /build/Main.o )
Linking Setup ...
updateAutotoolsGnuConfigScriptsPhase
configuring
configureFlags: --verbose --prefix=/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0 --libdir=$prefix/lib/$compiler --libsubdir=$abi/$libname --docdir=/nix/store/himlf5fg00791kfmmw3vcs3ia2y5l001-alpaca-netcode-0.1.0.0-doc/share/doc/alpaca-netcode-0.1.0.0 --with-gcc=gcc --package-db=/build/package.conf.d --ghc-options=-j16 +RTS -A64M -RTS --disable-split-objs --disable-library-profiling --disable-profiling --enable-shared --disable-coverage --enable-static --disable-executable-dynamic --enable-tests --disable-benchmarks --enable-library-vanilla --disable-library-for-ghci --ghc-option=-split-sections --extra-lib-dirs=/nix/store/bqknm9ar9p0d0l5215l826nx60bfrsip-ncurses-6.2/lib --extra-lib-dirs=/nix/store/llng85rn6an8k9krcxp3h9kgcz96ndwh-libffi-3.3/lib --extra-lib-dirs=/nix/store/22ljfc63ghv5n4y5s60035mv44dfi97q-gmp-6.2.1/lib
Using Parsec parser
Configuring alpaca-netcode-0.1.0.0...
Dependency base >=4.9 && <4.16: using base-4.14.1.0
Dependency bytestring -any: using bytestring-0.10.12.0
Dependency containers -any: using containers-0.6.2.1
Dependency flat -any: using flat-0.4.4
Dependency hashable -any: using hashable-1.3.0.0
Dependency network ==3.1.*: using network-3.1.1.1
Dependency network-run -any: using network-run-0.2.4
Dependency random -any: using random-1.1
Dependency stm -any: using stm-2.5.0.0
Dependency time -any: using time-1.9.3
Dependency alpaca-netcode -any: using alpaca-netcode-0.1.0.0
Dependency base >=4.9 && <4.16: using base-4.14.1.0
Dependency containers -any: using containers-0.6.2.1
Dependency random -any: using random-1.1
Dependency tasty -any: using tasty-1.4.1
Dependency tasty-hunit -any: using tasty-hunit-0.10.0.3
Source component graph:
    component lib
    component test:test dependency lib
Configured component graph:
    component alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        include base-4.14.1.0
        include bytestring-0.10.12.0
        include containers-0.6.2.1
        include flat-0.4.4-A7inmdBM3T91Q3uK8YFW19
        include hashable-1.3.0.0-6380ESuEbae43j1WfKKonK
        include network-3.1.1.1-F91Y9KbLQh5FWIKWxm6vtC
        include network-run-0.2.4-43DQlNNGteF7ZJVSWmJMuk
        include random-1.1-IcUwfYF3CGOHS82o6ktbLi
        include stm-2.5.0.0
        include time-1.9.3
    component alpaca-netcode-0.1.0.0-GYhzyDB4MQAz5poVVd7GJ-test
        include alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        include base-4.14.1.0
        include containers-0.6.2.1
        include random-1.1-IcUwfYF3CGOHS82o6ktbLi
        include tasty-1.4.1-7gC84J0Zddp6CBVtpkxhGW
        include tasty-hunit-0.10.0.3-6ygM5dNwmYLOSYF8mYZq7
Linked component graph:
    unit alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        include base-4.14.1.0
        include bytestring-0.10.12.0
        include containers-0.6.2.1
        include flat-0.4.4-A7inmdBM3T91Q3uK8YFW19
        include hashable-1.3.0.0-6380ESuEbae43j1WfKKonK
        include network-3.1.1.1-F91Y9KbLQh5FWIKWxm6vtC
        include network-run-0.2.4-43DQlNNGteF7ZJVSWmJMuk
        include random-1.1-IcUwfYF3CGOHS82o6ktbLi
        include stm-2.5.0.0
        include time-1.9.3
        Alpaca.NetCode=alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp:Alpaca.NetCode,Alpaca.NetCode.Advanced=alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp:Alpaca.NetCode.Advanced
    unit alpaca-netcode-0.1.0.0-GYhzyDB4MQAz5poVVd7GJ-test
        include alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        include base-4.14.1.0
        include containers-0.6.2.1
        include random-1.1-IcUwfYF3CGOHS82o6ktbLi
        include tasty-1.4.1-7gC84J0Zddp6CBVtpkxhGW
        include tasty-hunit-0.10.0.3-6ygM5dNwmYLOSYF8mYZq7
Ready component graph:
    definite alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        depends base-4.14.1.0
        depends bytestring-0.10.12.0
        depends containers-0.6.2.1
        depends flat-0.4.4-A7inmdBM3T91Q3uK8YFW19
        depends hashable-1.3.0.0-6380ESuEbae43j1WfKKonK
        depends network-3.1.1.1-F91Y9KbLQh5FWIKWxm6vtC
        depends network-run-0.2.4-43DQlNNGteF7ZJVSWmJMuk
        depends random-1.1-IcUwfYF3CGOHS82o6ktbLi
        depends stm-2.5.0.0
        depends time-1.9.3
    definite alpaca-netcode-0.1.0.0-GYhzyDB4MQAz5poVVd7GJ-test
        depends alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
        depends base-4.14.1.0
        depends containers-0.6.2.1
        depends random-1.1-IcUwfYF3CGOHS82o6ktbLi
        depends tasty-1.4.1-7gC84J0Zddp6CBVtpkxhGW
        depends tasty-hunit-0.10.0.3-6ygM5dNwmYLOSYF8mYZq7
Using Cabal-3.2.1.0 compiled by ghc-8.10
Using compiler: ghc-8.10.4
Using install prefix:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0
Executables installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/bin
Libraries installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/lib/ghc-8.10.4/aarch64-linux-ghc-8.10.4/alpaca-netcode-0.1.0.0-ClOsRIQdcRmE0OilEEVQkp
Dynamic Libraries installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/lib/ghc-8.10.4/aarch64-linux-ghc-8.10.4
Private executables installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/libexec/aarch64-linux-ghc-8.10.4/alpaca-netcode-0.1.0.0
Data files installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/share/aarch64-linux-ghc-8.10.4/alpaca-netcode-0.1.0.0
Documentation installed in:
/nix/store/himlf5fg00791kfmmw3vcs3ia2y5l001-alpaca-netcode-0.1.0.0-doc/share/doc/alpaca-netcode-0.1.0.0
Configuration files installed in:
/nix/store/9rpjfmvsg1w31s4mgkhmx2ch0zk8l531-alpaca-netcode-0.1.0.0/etc
No alex found
Using ar found on system at:
/nix/store/ppvq7f8cx4q4c7xjhp7ghs7pb5i8j7z9-binutils-2.35.1/bin/ar
No c2hs found
No cpphs found
No doctest found
Using gcc version 9.3.0 given by user at:
/nix/store/h8waayiawibip53lsvlc8fixfwlxqkpk-gcc-wrapper-9.3.0/bin/gcc
Using ghc version 8.10.4 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/ghc
Using ghc-pkg version 8.10.4 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.24.0 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.68 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/hpc
Using hsc2hs version 0.68.7 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/hsc2hs
Using hscolour version 1.24 found on system at:
/nix/store/p88488g435xzdj8wxgqsalkdhn0qh2c9-hscolour-1.24.4/bin/HsColour
No jhc found
Using ld found on system at:
/nix/store/w1d9m45rwallarkkkrzhcrc85afm1x81-binutils-wrapper-2.35.1/bin/ld.gold
No pkg-config found
Using runghc version 8.10.4 found on system at:
/nix/store/n5z7zr3jl1k96wcrrjqlc5bxdhjc1i69-ghc-8.10.4/bin/runghc
Using strip version 2.35 found on system at:
/nix/store/ppvq7f8cx4q4c7xjhp7ghs7pb5i8j7z9-binutils-2.35.1/bin/strip
Using tar found on system at:
/nix/store/a4lfl87zf306hid5qj0sgxy5g71nyjr8-gnutar-1.34/bin/tar
No uhc found
building
Preprocessing library for alpaca-netcode-0.1.0.0..
Building library for alpaca-netcode-0.1.0.0..
[1 of 6] Compiling Alpaca.NetCode.Internal.Common ( src/Alpaca/NetCode/Internal/Common.hs, dist/build/Alpaca/NetCode/Internal/Common.o, dist/build/Alpaca/NetCode/Internal/Common.dyn_o )
[2 of 6] Compiling Alpaca.NetCode.Internal.ClockSync ( src/Alpaca/NetCode/Internal/ClockSync.hs, dist/build/Alpaca/NetCode/Internal/ClockSync.o, dist/build/Alpaca/NetCode/Internal/ClockSync.dyn_o )
[3 of 6] Compiling Alpaca.NetCode.Internal.Client ( src/Alpaca/NetCode/Internal/Client.hs, dist/build/Alpaca/NetCode/Internal/Client.o, dist/build/Alpaca/NetCode/Internal/Client.dyn_o )
[4 of 6] Compiling Alpaca.NetCode.Internal.Server ( src/Alpaca/NetCode/Internal/Server.hs, dist/build/Alpaca/NetCode/Internal/Server.o, dist/build/Alpaca/NetCode/Internal/Server.dyn_o )
[5 of 6] Compiling Alpaca.NetCode.Advanced ( src/Alpaca/NetCode/Advanced.hs, dist/build/Alpaca/NetCode/Advanced.o, dist/build/Alpaca/NetCode/Advanced.dyn_o )
[6 of 6] Compiling Alpaca.NetCode   ( src/Alpaca/NetCode.hs, dist/build/Alpaca/NetCode.o, dist/build/Alpaca/NetCode.dyn_o )
Preprocessing test suite 'test' for alpaca-netcode-0.1.0.0..
Building test suite 'test' for alpaca-netcode-0.1.0.0..
[1 of 1] Compiling Main             ( test/Test.hs, dist/build/test/test-tmp/Main.o )
Linking dist/build/test/test ...
running tests
Running 1 test suites...
Test suite test: RUNNING...
alpaca-netcode
  Core:       OK (4.26s)
  UDP [NOCI]: test: Prelude.minimum: empty list
FAIL (8.14s)
    test/Test.hs:103:
    Expected at least 2000 auth worlds but client 1 got 750
  clientStop: OK (4.56s)

1 out of 3 tests failed (16.96s)
Test suite test: FAIL
Test suite logged to: dist/test/alpaca-netcode-0.1.0.0-test.log
0 of 1 test suites (0 of 1 test cases) passed.
builder for '/nix/store/z0b5lr22nv9xh1axp1kfwpv3z8g98qhh-alpaca-netcode-0.1.0.0.drv' failed with exit code 1

[DavidEichmann]

The failing test sets up a local UDP connection which can be flaky in a CI system. I've disabled it on github CI already. Perhaps you can disable it in your nix expression too (just filter out tests with [NOCI] in the name). FYI, the Core test does the same thing via Chans rather than an actual UDP socket.

[DavidEichmann]

... I could alternatively move that test into an integration test binary or something 🤔

[sternenseemann]

Is it possible to disable tests via the command line? It'd be great if I didn't have to patch anything.

[DavidEichmann]

@sternenseemann you can use cabal run test -- -p "!/[NOCI]/"

[DavidEichmann]

@sternenseemann did you manage to get things working?

[sternenseemann]

We are running tests with said filter now: https://github.com/NixOS/nixpkgs/blob/b27eaa18b47b77fbbf9f3bd7a17046b2d64b637b/pkgs/development/haskell-modules/configuration-common.nix#L1932-L1936