Cross-compilation failure with remote-iserv

[michivi]

Hi everyone,

I tried to cross-compile a Stack project for the Raspberry Pi (ARMv6l) but it fails with the following error:

---> Starting remote-iserv on port 6558
---| remote-iserv should have started on 6558
[         iserv-proxy] Created socket for 127.0.0.1:6558
[         iserv-proxy] Created socket for 127.0.0.1:6558
[         iserv-proxy] connected
[         iserv-proxy] Starting proxy
Listening on port 6558
[         iserv-proxy] fwdLoadCall: missing path: /nix/store/kf2cgy6gfrap9vrraxrx3jsscfnn8b1d-armv6l-unknown-linux-gnueabihf-ghc-8.6.5/lib/armv6l-unknown-linux-gnueabihf-ghc-8.6.5/ghc-prim-0.5.3/HSghc-prim-0.5.3.o
remote-iserv: Failed to lookup symbol: __sync_val_compare_and_swap_8


<no location info>: error:
    ghc: unable to load package `ghc-prim-0.5.3'

The Stack project uses GHC 8.6.5 with the latest version of Haskell-Nix (at the time I wrote this issue), and I had to set the following flags as mentioned in #313 :

packages.ghci.flags.ghci = true;
packages.libiserv.flags.network = true;

I tried cross-compiling the project to ARMv7l and the build ended properly.

Some investigations led me to believe that ghc-prim is using atomic builtin GCC functions that aren't defined for ARMv6l and were hence replaced with external function calls, which remote-iserv failed to find. If I am not mistaken, #387 did remove some of them (in atomic.c), but not the one defined in SMP.h, when a threaded RTS is used.

Have I overlooked something?

Thank you for your help

[angerman]

This sounds pretty similar to the stuff I did debug with @eamsden a while back... I don't really remember the reason anymore :-/

[eamsden]

@michivi What git revision of Haskell.nix did you use? Is it more recent than (i.e. does it include) the commit 563e18c?

This looks like one of the errors I was dealing with there: the main issue is that ghc doesn't build its core/boot libraries dynamically when nix sets it up for cross-compilation, so any other libraries that link to those (read, any libraries at all, pretty much) must also not be build dynamically. If you build with haskell.nix including that commit you should get static-only builds when cross-compiling to ARM.

[eamsden]

@angerman until my PR of a couple of weeks ago all that patching we wound up doing was languishing in input-output-hk/iohk-nix and never made it into haskell.nix, so if @michivi hasn't updated haskell.nix in the last week or two, ARM builds would be broken

[angerman]

@eamsden you are right, I was implicitly assuming he was using the most recent master. @michivi please provide the haskell.nix revision you are using!

[michivi]

@eamsden @angerman I used the most recent version of Haskell.nix I could at that time: efb9a07. I believe it does include commit 563e18c. I also did a test without that commit a few weeks ago, with exactly the same results. Aside from the custom flags I've put for ghci and libiserv in my Stack Nix project file, I'm pretty much using Haskell.nix as-is.

At first, I thought that was a problem in GHC (much like this 9886 GHC issue). But then, the patch ghc-8.6.4-prim-no-arm-atomics.patch removed those call, except for the one in SMP.h and I wondered if it was that call that brought all the other missing signatures. I thought perhaps using the perf-cross build flavor would remove the unused signatures but as you've mentioned in the associated PR, that is actually breaking qemu. I'm still looking for a solution, but my knowledge of GHC's internal is still limited. Thank you for your help!