Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv-darwin: now with 50% less LLVM! #87583

Merged
merged 1 commit into from May 15, 2020
Merged

Conversation

@Gaelan
Copy link
Contributor

Gaelan commented May 11, 2020

Motivation for this change

On Darwin, stdenv was building LLVM twice, with the only difference being that one used a stage 3 libxml2 and the other used one from stage 4. Unless I'm missing something (which is quite possible!), there's no reason to do this, and we can save a bunch of build time by just using the one LLVM.

I tested this by building stdenv and using it to build GNU hello, and also checked to make sure bootstrap-tools didn't end up in stdenv's closure.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@ofborg ofborg bot added the 6.topic: darwin label May 11, 2020
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020

cc @LnL7

@matthewbauer
Copy link
Member

matthewbauer commented May 11, 2020

Nice work!

@cole-h
Copy link
Member

cole-h commented May 11, 2020

This should probably be rebased against staging.

@ofborg eval

@FRidh FRidh added this to WIP in Staging via automation May 11, 2020
@Gaelan Gaelan changed the base branch from master to staging May 11, 2020
@Gaelan Gaelan force-pushed the Gaelan:single-llvm branch from 24f9d65 to 23ce1ee May 11, 2020
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020

Now against staging.

@LnL7
LnL7 approved these changes May 11, 2020
Copy link
Member

LnL7 left a comment

The goal here is to build all of the stdenv tooling from scratch without any references to the tools used to bootstrap it. Secondly some of the overrides are there to reduce the tree needed to do this, by default certain packages like sqlite would be included by python/cmake while they are not actually needed for example.

Assuming libxml2 from stage 3 and any other dependencies of that first llvm already don't have references anymore, then there shouldn't be any reason to build another one. If this is the case I wonder if we could even get rid of a stage.

Could you double check this?

$ nix-store -qR $(nix-build -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/vvk6w4jdwm5m5r1f6mlvqjx4p1lpmik8-bootstrap-stage4-binutils-wrapper-
/nix/store/xkl1gvhj841qq3m0abxlg9w06dpx11hh-bootstrap-stage4-clang-wrapper-
/nix/store/1rkmy85jsvqq9bnbcz4fwm246n4h9r3a-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build -A stdenv) | grep bootstrap

Staging automation moved this from WIP to Ready May 11, 2020
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020

$ git checkout single-llvm
Already on 'single-llvm'
Your branch is up to date with 'Gaelan/single-llvm'.

$ nix-store -qR $(nix-build . -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/9aa2rfiacjka8kvrg17ryyb6lhk5n62x-bootstrap-stage4-binutils-wrapper-
/nix/store/pm9jimjpvjpq0v9jsyswz5h35mx2bxi7-bootstrap-stage4-clang-wrapper-
/nix/store/rn1r46nyp4qfjhjbyysl91l0g1rv0q2l-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build . -A stdenv) | grep bootstrap

@Gaelan
Copy link
Contributor Author

Gaelan commented May 15, 2020

Is anything blocking this at this point?

@matthewbauer matthewbauer merged commit fb35227 into NixOS:staging May 15, 2020
16 checks passed
16 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="23ce1ee"; rev="23ce1eed34c97df283ad85ab40b8b33a6a8818eb"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
stdenv, stdenv.passthru.tests on aarch64-linux Success
Details
stdenv, stdenv.passthru.tests on x86_64-linux Success
Details
Staging automation moved this from Ready to Done May 15, 2020
@matthewbauer
Copy link
Member

matthewbauer commented May 15, 2020

Nope- merged to staging. If you could be on alert for any breakages from this that would be good though. Occasionally staging-next will get stuck because we don’t know why something broke.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.