Building neuron for aarch64 ?

[573]

I love to use neuron and do so on my laptop already.
What would be extra nice is to be able to use on my smartphone as well (where I have nix-on-droid installed already), so my plan was to:

• cross-build it on my laptop
• by using copy-closure or nix-copy transfer the built neuron closure to my smartphone

What others have found out and I have tested so far is that it is possible to - taking clay as an example as it builds:

nix-build -E 'with import <nixpkgs> {}; pkgsCross.aarch64-multiplatform-musl.haskellPackages.ghcWithPackages (p: [p.clay])'
nix-build "<nixpkgs>" -A pkgsCross.aarch64-multiplatform-musl.haskellPackages.clay

Are there chances to make the neuron build possible for any of the pkgsCross.aarch64-* targets as well ?

[srid]

@573 I don't know much about cross-building, but a PR is always welcome. Also see #45, which may be relevant.

[srid]

Oh and I'm planning to make a release to hackage soon. That should get it into stackage and nixpkgs.

[srid]

Thanks to @maralorn - neuron is now on nixpkgs, however note that it is an old version. nix-shell -p haskellPackages.neuron.

[srid]

@573 Have you managed to build neuron on aarch64?

Here's somebody else who is trying it: https://lobste.rs/s/7e7l03/how_i_publish_notes#c_vwmf0h

[573]

@srid I would like to build that on my phone as the latter is prepared to use a remote builder now.
How would I utilize the nixpkgs version ?
For now I have in my home-manager config:

# ~/.config/nixpkgs/program/haskell/default.nix
let unstable = import (fetchTarball https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz) {};
in { pkgs, config, ... }:
{
  home.packages = with unstable.pkgs.haskellPackages; [
    neuron
  ];
}

and

# ~/.config/nixpkgs/home.nix
{ pkgs ? import <nixpkgs> {},... }:
{
  imports = [ ./program/haskell ];
}

Build still fails though, see log (http://archive.vn/GVcZa).
On hydra there also seems to be no green build yet.

[maralorn]

@573 that is definitely a nixpkgs issue as our expression doesn't touch anything that affects the compilation of ghc.

Someone needs to open an issue with nixpkgs if that hasn't happened yet.

Besides that you could try if it works to use 8.8.3 instead. You can switch that in the default.nix by replacing hackagePackages with the appropriate specific version.

[573]

I tried again with ghc883 and the build although very long lasting (> 24hs) seems to do fine.
Currently there was a problem building jsaddle-dom with my remote builder.
I keep you updated.

[573]

No chance, even with profiling disabled

(justStaticExecutables (dontCheck (doJailbreak jsaddle-dom)))

[573]

I thought I am trying and building a bundle as suggested by @srid here.
But am not really getting any step forward as the bundle seems to be identical to the linux one and I can indeed run it on a x64 machine as opposed to an aarch64 one.
Any hints pls ?

[maralorn]

@573 I still believe your best bet is fixing (or getting someone to fix) the ghc884 build in nixpkgs. It might not help, but you should really try opening an issue in nixpkgs, if there isn't one. I would do it but I am limited to my phones keyboard right now.

[573]

@maralorn I guess I progressed a tiny step as I have the same error on github-actions now as on my phone earlier.
What still bothers me is this line:

qemu: Unsupported syscall: 276

EDIT: You're right in pointing to ghc884.
To have a workaround I could still try to let the build use ghc883 for the sake of having an aarch64 bundle.

[573]

I will continue my experiments here: https://github.com/573/neuron (https://github.com/573/neuron/tree/add-workflow-aarch64).

[maralorn]

@573 Have you tried building neuron on latest master? According to NixOS/nixpkgs#94649 this should be fixed.

[573]

@maralorn this is great news tryin asap. sorry for the delay.

[573]

@maralorn I built against the master of neuron as of yesterday and got this error (http://archive.vn/wip/FWxBg)

[maralorn]

@573 Is that error reproducible?

[maralorn]

I am sorry, I am out of my depth here. You have an issue with jira-wiki-markup-1.3.2 not building with an error that is either environmental e.g. OOM or something or a segfault. I have a few puzzle points here, but I am not sure what to make of it.

In nixpkgs neuron on aarch64 is right now broken on master because some dep has a bug on aarch64 and fails the test: https://hydra.nixos.org/build/125390340
So I am not sure if your issue would be reproduced by hydra or not. But this might be indicative for the hypothesis that your build error was only temporary?

Now jira-wiki-markup-1.3.2 only gets compiled as part of the neuron job, because of newer pandoc-types, etc. But jiri-wiki-markup-1.2.1 builds succesfully on aarch64 apparently: https://hydra.nixos.org/eval/1607860?filter=jira-wiki-markup&compare=1607838&full=#tabs-still-fail

Sadly I don‘t use nix on any aarch64 so I don‘t think I can help with this much more.

[573]

@maralorn I was indeed building locally on a rather old machine, OOM points in a reasonable direction:

nix-build -j4 --option system aarch64-linux --option sandbox false --extra-platforms aarch64-linux --expr 'import ./shell.nix { doCheck = false; }'

with my master in sync with upstream srid/neuron repo.

The reason I chose a local machine is that github-actions allow for certain amount of minutes for a job and one single build currently exceeds this.

Anyway thanks for the pointers.

[srid]

There is now a static.nix which is used to build static binaries on Linux. Maybe with a few changes that would also work for aarch64?

Try: nix-build static.nix.

[573]

@srid that is great news !
Still get some errors though (testsuite for GNU coreutils 8.31 is attempted).
EDIT: I'm adding

self:super: {
  coreutils = super.coreutils.overrideAttrs (old: {
    doCheck = false;
  });
}

to ~/.config/nixpkgs/overlays/neuron-static.nix and until now the build is running fine (currently building expat ;) )

[573]

FYI, the perl derivation of the build failing.

[573]

Shameless X-marketing, the nix-on-droid wiki now contains some more hints on installing NixOS in a (local) qemu vm to serve as a builder for aarch64 !

[srid]

Take a look at this redit thread -- someone used haskell.nix to cross compile a Haskell project for aarch64.

[573]

Latest, coreutils failing. With commit 8d9bc7341422a2346d8fd6dc35624723c6525f40 of neuron.

[573]

After some more findings I can

NIX_PATH=nixpkgs=http://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz nix-build --no-out-link -E 'let pkgsArm = import <nixpkgs> { config = {}; overlays = []; system = "aarch64-linux"; }; pkgsCross = import <nixpkgs> { overlays = [(self: super: { inherit (pkgsArm) cachix; })]; crossSystem = { config = "aarch64-unknown-linux-gnu"; }; }; in pkgsCross.cachix' --argstr system aarch64-linux --option sandbox false --builders 'ssh://nixos-shell aarch64-linux' -j0

RN to get a cachix executable targetting aarch64 directly from a binary cache.

$ file /nix/store/7qh1pjcbi930jcq2x9cwnnwg73cs3bip-cachix-0.6.0/bin/cachix
/nix/store/7qh1pjcbi930jcq2x9cwnnwg73cs3bip-cachix-0.6.0/bin/cachix: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/gczfd68r4x736zvq7aafd56hhwp91pjp-glibc-2.32-40/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, stripped

I would love to have the same for neuron.
It seems that neuron is not (yet) a toplevel nix package ATM though and thus I'm stuck as IDK how to do the same as in my expression listed here with i. e. haskellPackages.neuron.

EDIT: I tried

NIX_PATH=nixpkgs=http://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz nix-build --no-out-link -E 'let pkgsArm = import <nixpkgs> { config = {}; overlays = []; system = "aarch64-linux"; }; pkgsCross = import <nixpkgs> { overlays = [(self: super: { inherit (pkgsArm.haskellPackages) neuron; })]; crossSystem = { config = "aarch64-unknown-linux-musl"; }; }; in pkgsCross.haskellPackages.neuron' --argstr system aarch64-linux

but that gets me:

unpacking 'http://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz'...
error: attribute 'runtimeShell' missing, at /nix/store/fqcjcy4zhl54276g01wzv54lkk30njk3-nixexprs.tar.xz/pkgs/top-level/all-packages.nix:564:79

[573]

To reply my own question:

NIX_PATH=nixpkgs=http://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz nix-build --no-out-link -E 'let pkgsArm = import <nixpkgs> { config = {}; overlays = []; system = "aarch64-linux"; }; pkgsCross = import <nixpkgs> { overlays = [(self: super: { inherit (pkgsArm.haskellPackages) neuron; })]; crossSystem = { config = "aarch64-unknown-linux-musl"; }; }; in pkgsCross.neuron'

$ file /nix/store/nmd49gnn3i0w2c7kny7ijhd3s7hci0ab-neuron-1.0.0.0/bin/neuron
/nix/store/nmd49gnn3i0w2c7kny7ijhd3s7hci0ab-neuron-1.0.0.0/bin/neuron: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/gczfd68r4x736zvq7aafd56hhwp91pjp-glibc-2.32-40/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, stripped

Difference is I refer to my own closure's pkgsCross.neuron instead of (undefined) pkgsCross.haskellPackages.neuron.

That said, we found a general solution for the providing binary on aarch64 problem. I'll see if I can enhance that to even have a static version in the end for us.

[srid]

I wonder if your work can be leveraged to resume providing the x64 binary as well ...

By the way, the neuron in nixpkgs refers to v1, wheras master is on v2 and has deviated quite a bit.

[573]

Sounds interesting, if I could do something let me know. I'm not aware of the details regarding the x64 build though.

For static binaries there is a minor blocker (not when we use binfmt directly to spawn the build) currently: input-output-hk/haskell.nix#1107

[573]

Using the following command on my binfmt-enabled linux:

LC_ALL=C NIX_PATH=channel:nixos-20.09 nix-build '<nixpkgs>' --arg crossSystem '(import <nixpkgs> {}).lib.systems.examples.aarch64-multiplatform-musl' -A pkgsStatic.haskellPackages.neuron

the build started regularly but broke complaining about hslua (full log).

For a newer neuron version I still have to investigate:

LC_ALL=C NIX_PATH=channel:nixos-unstable nix-build '<nixpkgs>' --arg crossSystem '(import <nixpkgs> {}).lib.systems.examples.aarch64-multiplatform-musl' -A pkgsStatic.haskellPackages.neuron

complains about attribute 'runtimeShell' missing (...) top-level/all-packages.nix.