pkgsMusl.stdenv: fix build on aarch64 #169764
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The C++ compiler in our musl bootstrap for aarch64 is too old to build the latest version of patchelf, so introduce a package for the most recent version it's capable of building that we can use to get a new bootstrap.
The C++ compiler in our musl bootstrap for aarch64 is too old to build the latest version of patchelf, so we need to use the latest version that builds with that compiler to get a new bootstrap.
I suspect this issue is caused by the very old version of diffutils we have in the aarch64+musl bootstrap tools, so hopefully we can re-enable the tests once we have newer bootstrap tools.
alyssais
force-pushed
the
aarch64-musl-bootstrap
branch
from
908ae6f
to
a30237c
Compare
|
(Another part of this will be getting Hydra to actually build bootstrap tarballs for Musl, but that's for another PR.) |
13 tasks
alyssais
added
the
6.topic: bootstrap
sternenseemann
approved these changes
Apr 22, 2022
Mindavi
approved these changes
Apr 22, 2022
|
Woohoo, time to fix it :) |
alyssais
added a commit
to alyssais/nixpkgs
that referenced
this pull request
Jan 24, 2024
The aarch64 musl bootstrap tools are woefully outdated. Just getting them to the point of being able to build new versions of themselves required a number of hacks[1] that can be reverted once we have new bootstrap tools, and before that it stdenv didn't even build for the preceding three years. [1]: NixOS#169764 So, following the script established by previous bootstrap tools updates: Files came from this Hydra build: https://hydra.nixos.org/build/246470544 …which used nixpkgs revision dd5621d to instantiate: /nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv …and then built: /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools I downloaded these files from Hydra and prefetched them into the Nix store with the following commands: STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" nix --extra-experimental-features nix-command store prefetch-file \ file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz) nix --extra-experimental-features nix-command store prefetch-file --executable \ file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox) These commands produced the following output: warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE='). warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs='). I used the hashes from the output above to create the fetchurl invocation which is part of this commit. I then started the bootstrap with the following command: nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello As @lovesegfault requested, here are the the sha256sums of all the on-server components for extra verification: sha256sum /nix/store/${STOREPATH}/on-server/* …which produced the following output: 658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01 /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz 20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
alyssais
added a commit
to alyssais/nixpkgs
that referenced
this pull request
Jan 24, 2024
The aarch64 musl bootstrap tools are woefully outdated. Just getting them to the point of being able to build new versions of themselves required a number of hacks[1] that can be reverted once we have new bootstrap tools, and before that it stdenv didn't even build for the preceding three years. [1]: NixOS#169764 So, following the script established by previous bootstrap tools updates: Files came from this Hydra build: https://hydra.nixos.org/build/246470544 …which used nixpkgs revision dd5621d to instantiate: /nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv …and then built: /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools I downloaded these files from Hydra and prefetched them into the Nix store with the following commands: STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" nix --extra-experimental-features nix-command store prefetch-file \ file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz) nix --extra-experimental-features nix-command store prefetch-file --executable \ file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox) These commands produced the following output: warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE='). warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs='). I used the hashes from the output above to create the fetchurl invocation which is part of this commit. I then started the bootstrap with the following command: nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello As @lovesegfault requested, here are the the sha256sums of all the on-server components for extra verification: sha256sum /nix/store/${STOREPATH}/on-server/* …which produced the following output: 658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01 /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz 20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
alyssais
added a commit
that referenced
this pull request
Jan 25, 2024
The aarch64 musl bootstrap tools are woefully outdated. Just getting them to the point of being able to build new versions of themselves required a number of hacks[1] that can be reverted once we have new bootstrap tools, and before that it stdenv didn't even build for the preceding three years. [1]: #169764 So, following the script established by previous bootstrap tools updates: Files came from this Hydra build: https://hydra.nixos.org/build/246470544 …which used nixpkgs revision dd5621d to instantiate: /nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv …and then built: /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools I downloaded these files from Hydra and prefetched them into the Nix store with the following commands: STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" nix --extra-experimental-features nix-command store prefetch-file \ file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz) nix --extra-experimental-features nix-command store prefetch-file --executable \ file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox) These commands produced the following output: warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE='). warning: you did not specify '--add-root'; the result might be removed by the garbage collector Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs='). I used the hashes from the output above to create the fetchurl invocation which is part of this commit. I then started the bootstrap with the following command: nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello As @lovesegfault requested, here are the the sha256sums of all the on-server components for extra verification: sha256sum /nix/store/${STOREPATH}/on-server/* …which produced the following output: 658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01 /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz 20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
13 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
As identified in #168207, the musl stdenv has been broken on aarch64 for a very long time (more than three years!). The main problem is that the bootstrap tarball is severely outdated. But in order to have Hydra generate a new one, we need to give it something it can build with the old one as a starting point.
To do that, downgrade patchelf to a version that can be compiled with the current aarch64+musl bootstrap C++ compiler, and disable the tests for coreutils since they run into a problem with the bootstrap tarball's diff program.
Once Hydra has generated an up to date bootstrap tarball, we can update the aarch64+musl stdenv to use that, and then revert these hacks.
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)nixos/doc/manual/md-to-db.shto update generated release notes