pkgsCross.aarch64-multiplatform.ubootQemuAarch64 fails to build on aarch64-darwin

[mstone]

Build failure

This may be a tall order, but ultimately, I'd like

nix build nixpkgs#pkgsCross.aarch64-multiplatform.ubootQemuAarch64

to succeed on aarch64-darwin, to make it easy to do linux kernel development on M1/... chips under QEMU.

Situation

Today, this command fails initially due to #137877, which is now fixed in staging-next, in commit e9f52e8.

Continuing with nixpkgs from that commit:

nix build nixpkgs/e9f52e843edb26059593be60d5c8cb237dff87ba#pkgsCross.aarch64-multiplatform.ubootQemuAarch64

reveals new errors that look like:

       > sh: line 1: sw_vers: command not found
       > sh: line 1: [: too many arguments
       > sh: line 1: gcc: command not found
       > sh: line 1: sw_vers: command not found
       > sh: line 1: sw_vers: command not found
       > sh: line 1: [: too many arguments
       >   HOSTCC  scripts/basic/fixdep
       > sh: line 1: gcc: command not found
       > make[1]: *** [scripts/Makefile.host:95: scripts/basic/fixdep] Error 127
       > make: *** [Makefile:492: scripts_basic] Error 2

I think due to u-boot's Makefile calling sw_vers to decide what compiler/linker flags to pass to different versions of Apple's C toolchain: https://source.denx.de/u-boot/u-boot/-/blob/master/Makefile#L308-338.

The fix for this issue, I think, is to add darwin.DarwinTools to nativeBuildInputs or similar.

Continuing with that change:

nix build --impure --expr 'with builtins; with (getFlake "nixpkgs/e9f52e843edb26059593be60d5c8cb237dff87ba"); with (getAttr currentSystem legacyPackages); pkgsCross.aarch64-multiplatform.ubootQemuAarch64.overrideAttrs (old: { nativeBuildInputs = old.nativeBuildInputs ++ [ darwin.DarwinTools ]; })'

we get a new error about gcc not being found:

       > tools/zynqmp_pm_cfg_obj_convert.py: interpreter directive changed from "#!/usr/bin/env python3" to "/nix/store/s8wxan6fkack8d2f5nj983qwpkzrzhhf-python3-3.10.8-env/bin/python3"
       > patching script interpreter paths in arch/arm/mach-rockchip
       > arch/arm/mach-rockchip/make_fit_atf.py: interpreter directive changed from "#!/usr/bin/env python3" to "/nix/store/s8wxan6fkack8d2f5nj983qwpkzrzhhf-python3-3.10.8-env/bin/python3"
       > updateAutotoolsGnuConfigScriptsPhase
       > configuring
       > sh: line 1: gcc: command not found
       >   HOSTCC  scripts/basic/fixdep
       > sh: line 1: gcc: command not found
       > make[1]: *** [scripts/Makefile.host:95: scripts/basic/fixdep] Error 127
       > make: *** [Makefile:492: scripts_basic] Error 2

This is actually due to the need to build Kconfig itself as part of configurePhase and can, in principle, be solved either by making gcc available or by using clang to build Kconfig, with something like either

nix build --impure --expr 'with builtins; with (getFlake "nixpkgs/e9f52e843edb26059593be60d5c8cb237dff87ba"); with (getAttr currentSystem legacyPackages); pkgsCross.aarch64-multiplatform.ubootQemuAarch64.overrideAttrs (old: { nativeBuildInputs = old.nativeBuildInputs ++ [ gcc darwin.DarwinTools ]; })'

or

nix build --impure -L --expr 'with builtins; with (getFlake "nixpkgs/e9f52e843edb26059593be60d5c8cb237dff87ba"); with (getAttr currentSystem legacyPackages); (pkgsCross.aarch64-multiplatform.ubootQemuAarch64.override (old: { defconfig = "CC=clang HOSTCC=clang " + old.defconfig; })).overrideAttrs (old: { nativeBuildInputs = old.nativeBuildInputs ++ [ darwin.DarwinTools ]; })'

Unhappily, neither of these attempted fixes works in the end, for different reasons but in similar ways across many variations of HOSTCC=clang / HOSTCC=gcc:

1. the gcc-based version fails later on in buildPhase with errors that look like

...
duplicate symbol '___start_image_type' in:
    tools/aisimage.o
    tools/lib/rsa/rsa-mod-exp.o
duplicate symbol '___stop_image_type' in:
    tools/aisimage.o
    tools/lib/rsa/rsa-mod-exp.o
duplicate symbol '___start_image_type' in:
    tools/aisimage.o
    tools/fit_info.o
duplicate symbol '___stop_image_type' in:
    tools/aisimage.o
    tools/fit_info.o
ld: 84 duplicate symbols for architecture arm64
collect2: error: ld returned 1 exit status
make[1]: *** [scripts/Makefile.host:104: tools/fit_info] Error 1
make: *** [Makefile:1875: tools] Error 2

2. whereas the clang version fails with errors that look like

  HOSTCC  tools/lib/crc32.o
In file included from <built-in>:1:
In file included from ././include/compiler.h:23:
include/errno.h:28:5: error: function-like macro 'CONFIG_IS_ENABLED' is not defined
#if CONFIG_IS_ENABLED(ERRNO_STR)
    ^
In file included from <built-in>:1:
In file included from ././include/compiler.h:24:
In file included from include/stdlib.h:9:
In file included from include/malloc.h:369:
In file included from include/linux/string.h:21:
In file included from ./arch/arm/include/asm/string.h:4:
In file included from include/config.h:4:
...
include/stdio.h:53:20: error: expected function body after function declarator
int __printf(2, 3) fprintf(int file, const char *fmt, ...);
                   ^
In file included from <built-in>:1:
In file included from ././include/compiler.h:49:
In file included from /nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/machine/endian.h:37:
In file included from /nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/arm/endian.h:75:
In file included from /nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/sys/_endian.h:130:
In file included from /nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/libkern/_OSByteOrder.h:80:
/nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/libkern/arm/OSByteOrder.h:14:1: error: unknown type name 'uint16_t'
uint16_t
^
/nix/store/0y7h0mx2p7v1hn5hzqfms8gqjmcwf1ld-libSystem-11.0.0/include/libkern/arm/OSByteOrder.h:16:2: error: unknown type name 'uint16_t'
        uint16_t        _data
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[1]: *** [scripts/Makefile.host:95: tools/gen_eth_addr] Error 1
make: *** [Makefile:1875: tools] Error 2
make: *** Waiting for unfinished jobs....
  OFCHK   .config

Finally, digging one step deeper on the HOSTCC=gcc family of errors: I think that part of the problem on this branch of the tree of attempted fixes is that nixpkgs on Darwin secretly wraps Apple's Mach-O ld, which AIUI can no longer be configured to tolerate multiply-defined symbols in the same way that GNU ld can?

(Specifically: Darwin ld does not accept the --accept-multiple-definition / -z multdef flags that GNU ld does and further, recent versions of Darwin ld have obsoleted the previous workaround that U-Boot natively uses, which is to pass -multiply_defined suppress when it detects that Darwin's ld is in use, which I think it does not do successfully in this setup?)

Notify maintainers

@bartsch @dezgeg @samueldr @lopsided98

Thanks

Finally, regards + thanks in advance for any thoughts / suggestions; I know this is a rather finicky thing to try to get to work!