Multiple packages fail to build due to tests using locks

[unode]

Issue description

libidn includes a test: test-lock that never finishes.
libunistring includes a similar test lt-test-lock that behaves the same.

If a maximum build time is enforced or if these tests are killed manually, installation errors due to a test-suite failure.

This happens on a CentOS system running nix on a custom root (/share/nix) directory.
Compilation happens on /dev/shm while /share/nix lives in an NFS based filesystem.
Installation is performed by nix-daemon. The entire setup has no root access.

Some output from nix-env:

...
=================================================
   GNU Libidn 1.33: lib/gltests/test-suite.log
=================================================

# TOTAL: 34
# PASS:  33
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test-lock
===============

Starting test_lock ... OK
Starting test_rwlock ...
============================================================================
Testsuite summary for GNU Libidn 1.33
============================================================================
# TOTAL: 34
# PASS:  33
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See lib/gltests/test-suite.log
Please report to bug-libidn@gnu.org
============================================================================
make[5]: *** [Makefile:2125: test-suite.log] Error 1
make[5]: Leaving directory '/dev/shm/nix-build-libidn-1.33.drv-0/libidn-1.33/lib/gltests'
make[4]: *** [Makefile:2233: check-TESTS] Error 2
make[4]: Leaving directory '/dev/shm/nix-build-libidn-1.33.drv-0/libidn-1.33/lib/gltests'
make[3]: *** [Makefile:2560: check-am] Error 2
make[3]: Leaving directory '/dev/shm/nix-build-libidn-1.33.drv-0/libidn-1.33/lib/gltests'
make[2]: *** [Makefile:2017: check-recursive] Error 1
make[2]: Leaving directory '/dev/shm/nix-build-libidn-1.33.drv-0/libidn-1.33/lib/gltests'
make[1]: *** [Makefile:2562: check] Error 2
make[1]: Leaving directory '/dev/shm/nix-build-libidn-1.33.drv-0/libidn-1.33/lib/gltests'
make: *** [Makefile:1333: check-recursive] Error 1

Steps to reproduce

Build any package that depends on libidn

Technical details

• System: CentOS release 6.5 - 2.6.32-431.11.2.el6.x86_64
• Nix version: nix-env (Nix) 1.11.7
• Nixpkgs version: "17.09pre-git"
• Sandboxing enabled: No (running on non-nixos without root access)

[fadenb]

Can confirm this issue on NixOS. Albeit I only ever have encountered this issue on one specific system and was unable to reproduce it when I wanted to.

[unode]

I can reproduce it reliably.
If there's any additional information that might be of use to address this let me know.

[dezgeg]

Do you have lots of cores? https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26441

[unode]

8 cores only, but might be related if there's some sort of race condition in the test that can deadlock.

[dtzWill]

I have had the same problem on my NUMA 32core machine, that test is a mess.

More recent thread, summary of some problems: https://lists.gnu.org/archive/html/bug-gnulib/2017-01/msg00009.html

Commit history shows various problems and attempts to fix them through early 2017:
https://github.com/coreutils/gnulib/commits/master/tests/test-lock.c

I'm not sure if it helps, but I /seem/ to recall that the CentOS kernel also has some bug that causes this or perhaps makes it worse. Unfortunately I'm not finding information on that but that's what my memory says so thought I'd share O:).

[vcunat]

Well, if they're flaky, we can disable them for now, in all packages where they are. Any better suggestions? (It would be a stdenv rebuild.)

[Ekleog]

(triage) I've run three times nix-build '<nixpkgs>' -A libidn --check successfully; does this issue still happen or can this issue be closed?

[unode]

I'm having troubles reproducing it too. Both libidn and libunistring now pass.
Wondering if the kernel issues @dtzWill mentioned have been solved in the meanwhile...

Leaving the final decision to the higher ups.