GHC RTS linker errors when upgrading from 8.8 to 8.10

[patrickt]

Hey, everyone! 👋 @rewinfrey and I have been running into some strange behavior on our CI boxes, and we were wondering if anyone has any insights or advice. When we upgraded our targeted GHC version (within rules_haskell_toolchains), we started seeing a bunch of linker errors, of the form

ERROR: /home/runner/.cache/bazel/_bazel_runner/fe31aefe607f1517e4d186a8036a379a/external/stackage/BUILD.bazel:1773:22: HaskellCabalLibrary @stackage//:raw-strings-qq failed (Exit 1) cabal_wrapper failed: error executing command bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:raw-strings-qq raw-strings-qq-1.1 true external/stackage/raw-strings-qq-1.1/Setup.hs external/stackage/raw-strings-qq-1.1 ... (remaining 9 argument(s) skipped)
35

36
Use --sandbox_debug to see verbose messages from the sandbox
37
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o: In function `_start':
38
(.text+0x20): undefined reference to `main'
39
../../../bazel-out/k8-fastbuild/bin/external/stackage/raw-strings-qq-1.1/tmp04mp_m0g/build/Text/RawString/QQ.dyn_o:(.text+0xb1): undefined reference to `newCAF'
40
../../../bazel-out/k8-fastbuild/bin/external/stackage/raw-strings-qq-1.1/tmp04mp_m0g/build/Text/RawString/QQ.dyn_o:(.text+0xc1): undefined reference to `stg_bh_upd_frame_info'
41
../../../bazel-out/k8-fastbuild/bin/external/stackage/raw-strings-qq-1.1/tmp04mp_m0g/build/Text/RawString/QQ.dyn_o:(.text+0x104): undefined reference to `stg_upd_frame_info'
42
../../../bazel-out/k8-fastbuild/bin/external/stackage/raw-strings-qq-1.1/tmp04mp_m0g/build/Text/RawString/QQ.dyn_o:(.text+0x144): undefined reference to `stg_upd_frame_info’

Note that the base and template-haskell packages also run into this problem, if you scroll down all the way. A build with the sandbox_debug option didn’t shed a huge amount of light on the situation: it seems to be mounting and remounting a bunch of directories and making an occasional call to sigaction, which fails. However, I don’t see a smoking gun in the scrollback.

Things we’ve tried thus far:

• upgrading rules_haskell and bazel to the newest/specified versions
• seeing if test vs. build made a difference (no)
• downgrading to 8.8.3 but keeping everything the same (it works fine with 8.8.3, but we’d like to use 8.10 if possible)

To Reproduce
We haven’t been able to reproduce this on a local machine, unfortunately; the best way I know how to reproduce this is to fork semantic and push a branch that bumps the GHC version in the WORKSPACE. The PR on which we’re seeing this is github/semantic#620.

Environment

• OS name + version: Ubuntu 18.04.5, on a GitHub Actions runner
• Bazel version: 3.3.1
• Version of the rules: HEAD (aabeedc18f5e5db030ca1aa0c10a7dc14e4a4a55)

Additional context
As always, I appreciate your time and attention!

[robinbb]

@patrickt I will try to repro.

[mboes]

I can reproduce. Starting from a stock Debian stable (buster) image with Bazel installed.

[mboes]

Minimal repro:

$ cd rules_haskell
$ bazel build @stackage//:void

after having edited the test GHC version to be 8.10.1 instead of 8.8.3.

The linker error appears on Debian buster with GHC bindists, and on NixOS with a Nixpkgs-sourced toolchain. And it appears at link time on a very simple package. So it's a general "support for GHC 8.10" issue.

[mboes]

@patrickt looks like it's the same issue biting again: #1327 (comment).

[robinbb]

@patrickt I have repro'd as well, using github/semantic. Your current 'master' is broken, right? In other words, it is not the case that you want to move to 8.10 - you have already moved from 8.8 to 8.10, and therefore your 'master' does not build?

[robinbb]

This workaround for github/semantic builds for me:

diff --git a/WORKSPACE b/WORKSPACE
index c0df66213..b3a8b52c6 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -38,7 +38,7 @@ load(
 # Download a GHC binary distribution from haskell.org and register it as a toolchain.
 rules_haskell_toolchains(
     locale = "en_US.UTF-8",
-    version = "8.10.1",
+    version = "8.8.3",
 )

I recognize that is not what is desired. You want GHC 8.10. Looking into that...

[robinbb]

Oh, I see that github/semantic#595 was resolved by using the the fix for that on rules_haskell. That's good, but it means that using an earlier version of rules_haskell in github/semantic is not an option for a workaround in this instance. A proper fix is required.

[aherrmann]

Does #1424 fix this issue?

[robinbb]

@patrickt Is this fixed for you?