Setup.hs must be compiled with -threaded

[orlitzky]

I ran into Gentoo bug #537500 with the test suite for ShellCheck. I asked Haskell-Cafe about it, and Thomas Tuegel told me to report it here. So here you go!

I was able to reproduce the issue with the latest Cabal master branch.

[trofi]

slightly more info with -debug +RTS -Ds runtime:

Test suite test-shellcheck: RUNNING...
cap 0: created thread 4
cap 0: thread 1 stopped (yielding)
cap 0: running thread 4 (ThreadRunGHC)
cap 0: thread 4 stopped (blocked on a read operation)
        thread    4 @ 0x7ffff5c27390 is blocked on read from fd 3 (TSO_DIRTY)
scheduler: checking for threads blocked on I/O
cap 0: running thread 1 (ThreadRunGHC)
cap 0: thread 1 stopped (suspended while making a foreign call)
cap 0: running thread 1 (ThreadRunGHC)
'dist/build/test-shellcheck/test-shellcheck'
cap 0: thread 1 stopped (suspended while making a foreign call)
^C
Program received signal SIGINT, Interrupt.
0x00007ffff7bcbe9c in __libc_waitpid (pid=30078, stat_loc=0x7fffffff9cbc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
31      ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
(gdb) i th
  Id   Target Id         Frame 
* 1    Thread 0x7ffff7fda740 (LWP 30072) "setup" 0x00007ffff7bcbe9c in __libc_waitpid (pid=30078, stat_loc=0x7fffffff9cbc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
(gdb) bt
#0  0x00007ffff7bcbe9c in __libc_waitpid (pid=30078, stat_loc=0x7fffffff9cbc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
#1  0x00000000009d4c74 in waitForProcess ()
#2  0x00000000009d3718 in s54b_info ()
#3  0x0000000000000000 in ?? ()

waitpid() stuck and does not get unblocked by VTALARM (signals stop hppening as ./setup does nothing).

[ttuegel]

I strongly suspect this is yet another lazy IO bug. At this point, that deserves its own label.

[bennofs]

I think the cause of this problem is that rawSystemIOWithEnv uses ẁaitForProces which being an FFI call blocks all other haskell threads if using the non-threaded runtime. The runTest function in Distribution/Simple/Test/ExeV10.hs however depends on threads to read the pipe which is passed to the process for outputting. If the pipe fills up (because the read thread never runs), the test suite will block.

[ttuegel]

This is a different flavor of #2489. I am testing the recommended fix for that now. I'll apply it here once everything checks out.

[nomeata]

Is there a better work-around than building with -threaded, which is not available on all architectures?

[ttuegel]

@nomeata No, the Setup.hs executable is designed to only be compiled with -threaded. If that option is not available on all architectures, I will need to remove the --show-details=streaming option from the test command. I guess we can keep that option in cabal-install, which already requires -threaded.

[nomeata]

It might make sense to keep Setup.hs simple and reliable, e.g. for automatic builders, and do fancy stuff in cabal-install.

If it is related to --show-details=streaming, maybe --show-details=always could be implemented so that it works without threading?

[ttuegel]

Yes, that's how it was originally implemented. In fact, if Cabal's log files aren't needed, we can even just pipe the test output directly to stdout. I suspect this is the case for most automatic builders, which will have their own logging facilities. Threading is only required to both keep logs and print real-time test output.

[nomeata]

I see a few test suite failures on various packages, and due to this bug I’m not sure whether to blame cabal or the actual package at hand. I’d be grateful for a patch, especially if it is simple and can be backported to GHC-7.10.

[nomeata]

I’m confused now. I tried to add a test case to Cabal to reproduce the problem, but I’m failing to to do so. Moreover, it seems that there, Setup is compiled without -threaded (see compileSetup in PackageTester.hs).

Maybe the runtime works fine without -threaded, but on some architectures, it does not manage to switch threads?

[nomeata]

Ok, I could reproduce it now, with large enough output. Preparing a pull request with the updated output.

I suggest to go this, less ambitious but more reliable route:

• --show-details=never: Write output directly to the log, i.e. pass the handle to the log file to the test program.
• --show-details=always: Write output directly to the log, i.e. pass the handle to the log file to the test program. Afterwards read that file and print it.
• --show-details=failure: Write output directly to the log, i.e. pass the handle to the log file to the test program. Afterwards read that file and print it, if there was a failure
• --show-details=streaming: Write output directly to stdout. Do not write a log file.

The last bit is a slight loss of functionality, but avoids any threading or pipe foo in Cabal, and furthermore fixes #2911 en passant.