Enable cross-compilation between both Darwin platforms #256590
Conversation
reckenrode
force-pushed
the
darwin-cross
branch
from
af2c4e7
to
dec9a32
Compare
ofborg
bot
added
6.topic: cross-compilation
reckenrode
force-pushed
the
darwin-cross
branch
from
dec9a32
to
94b24ce
Compare
|
The LLVM 11 build failed in compiler-rt on aarch64-darwin. I could have reverted back to #123524, but it would be really nice to have parity between both architectures. Fortunately, I was able to fix it by backporting some of the changes from the compiler-rt 16 derivation. It’s building again, and hopefully it should finish by this evening. |
…es (#56959) We want to get newer stuff in the devshell, but keep a stabler pin for the static binaries (until darwin stdenv stabilizes, most notably the big brain work over at NixOS/nixpkgs#256590) ## Test plan N/A nix stuff :clueless:
reckenrode
requested review from
RaitoBezarius and
Ericson2314
as code owners
|
The clang 11 builds completed successfully after I added the fix for compiler-rt. Setting this to ready to review. |
reckenrode
force-pushed
the
darwin-cross
branch
from
94b24ce
to
1a2d7cd
Compare
reckenrode
force-pushed
the
darwin-cross
branch
from
1a2d7cd
to
a8d8c76
Compare
|
@toonn I believe I’ve addressed all the feedback. I tested static and dynamic cross builds to both x86_64-darwin and aarch64-darwin. I also confirmed Wine builds with the clang 16 stdenv (it does not build with the clang 11 one currently). |
|
darwin-cross is at commit a8d8c76
pkgsCross.aarch64-darwin.llvm.log
|
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
I finally got a chance to dig into the LLVM issue. I’ve been testing on my clang 16 branch, so I had to rebuild this branch for clang 11 to test and confirm, and that took a little bit. This is the issue. nixpkgs/pkgs/development/compilers/llvm/11/llvm/default.nix Lines 267 to 271 in a68655c LLVM needs a native compiler when doing a cross build. That is set up correctly, but the native compiler also needs certain dependencies propagated to it (such as the path to However, I do have a patch, but I want to make sure it applies and works across all of the LLVM versions. I’ll submit the PR (probably tomorrow morning EDT) for it once those finish building (for both aarch64-darwin and x86_64-darwin cross). |
|
I opened #260543 for the LLVM cross-compilation fix. |
* Separate Csu and libresolv into separate stage 0 packages to mirror how they are organized normally; * Make bintools overridable for consistency with non-bootstrap bintools; * Propagate libresolv through the assertions to the final output; and * Update Libsystem assertions to reflect the fact it will be building with `stdenvNoCC` on both x86_64-darwin and aarch64-darwin.
Libsystem depends (directly and indirectly) on a few packages that require clang to build. This stdenv uses a Libsystem that omits these packages, so the one used by the rest of nixpkgs can include them.
The SDK frameworks don’t need a compiler in the general case, so don’t use one unnecessarily. This avoids an infinite recursion when cross-compiling to x86_64-darwin.
configd does not require a full Libsystem to build. Break an infinite recursion by using the bootstrap stdenv.
Csu only needs to build stub objects, so it only needs to use the bootstrap stdenv. This breaks an infinite recursion when cross-compiling to x86_64-darwin.
GCC has been updated not to pass `-rpaths` because the updated ld64 won’t accept them for older deployment targets regardless of what the target is of the object files, so this workaround is no longer needed.
Break an infinite recursion by using the bootstrap stdenv, which disables libresolv support in Libsystem.
* Always use `stdenvNoCC` to build Libsystem. This is needed to prevent an infinite recursion when cross-building Libsystem; * Drop ncurses from Libsystem. While upstream does ship ncurses headers, everything in nixpkgs should be using ncurses from nixpkgs and not using upstream’s headers. This fixes an infinite recursion because ncurses still needs to be build to produce an ncurses^dev output; * Allow the stubs to be conditionally included in Libsystem. This is needed to build a bootstrap stdenv for building libresolv and Csu; * Symlink libresolv instead of copying it into Libsystem’s output. It’s not clear why this is necessary anymore, and using a stdenv with `install_name_tool` causes an infinite recursion when cross-building.
* Break an infinite recursion by determining the target prefix from the platform config. Referencing `stdenv.cc.bintools.targetPrefix` causes an infinite recursion because xnu is a dependency of Libsystem; * Add the host compiler as a dependency for the headers-only build, which needs to build tools it will use at build time; and * Add `migcc`, which provides a target compiler for use with `mig`, which needs to invoke clang for the target system during the build.
Break an infinite recursion with CF due to its depending on libc++ and compiler-rt.
With the infinite recursion broken when including darwin.CF in the stdenv, both regular cross and static builds can use the same `extraBuildInputs` override.
This ensures it is properly dropped when cross-compiling.
abseil-cpp will try to build for both architectures on Darwin. Not sure why this works with clang 16, but clang 18 fails when x86_64 flags are used when building for aarch64-darwin.
b2 tries to invoke the linker without a prefix, which fails.
The http-parser `Makefile` excepts the variable to be `PLATFORM`. Otherwise, it runs `uname -s` and detects the wrong target platform when cross-compiling to Linux from Darwin.
libpsl needs to link against gettext, which requires it be included as a build input even if gettext is already propagated as a native build input.
Actual static builds using `-static` are not supported, but Darwin can still do mostly static.
Taken from NixOS/nixpkgs#256590
Description of changes
While cross-compilation from x86_64-darwin to aarch64-darwin already works, going the other way runs into issues with the source-based SDK. This PR resolves those issues and enables cross-compilation to x86_64-darwin. It should now be possible to build any of the following scenarios from a Darwin host:
I have tried to keep the changes required small and limited. While working on this PR, I found a few derivations that did not build correctly when cross-compiling. Fixing those was outside the scope of this PR unless they were required by the bootstrap or are part of the cross-stdenv.
Known breakage: brotli (references a dylib from the build environment), Wine (has a
buildInputin one of its phase scripts), DarwinTools (needs modifications for cross-compilation). MoltenVK is probably also broken.Regarding Linux: I briefly looked at whether this would work on Linux. There’s work needing to be done to make certain required tools like bootstrap_cmds work on Linux. Expanding cross-compilation to Linux is outside the scope of this PR.
Note: My tests were done using my clang 16 branch. I have successfully evaluated the clang 11 branch used for this PR, but it will take overnight to build. I wanted to open this PR to get it out there for review, so I am opening it as a draft. If all goes well, I’ll flip this to ready tomorrow (Friday, 2023-09-22).
Resolved Issues
This PR should close the following issues:
Prerequisite PRs
Related PRs
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)