Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

treewide: change fetchCargoTarball default to opt-out #79978

Merged
merged 1 commit into from Feb 14, 2020

Conversation

@bhipple
Copy link
Contributor

bhipple commented Feb 13, 2020

Changes the default fetcher in the Rust Platform to be the newer
fetchCargoTarball, and changes every application using the current default to
instead opt out.

This commit does not change any hashes or cause any rebuilds. Once integrated,
we will start deleting the opt-outs and recomputing hashes.

See #79975 for details.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@bhipple bhipple requested review from andir and jonringer as code owners Feb 13, 2020
@bhipple bhipple requested a review from zimbatm Feb 13, 2020
@bhipple

This comment has been minimized.

Copy link
Contributor Author

bhipple commented Feb 13, 2020

There are no manual changes in this PR; the diff is generated with the following script, which you're welcome to review in addition to the actual diff:
https://github.com/bhipple/dotfiles/blob/master/bin/fix-rust-p2.sh

In order to check for diffs, this script can be run to build a number of packages on master and then again after running the above script:
https://github.com/bhipple/dotfiles/blob/master/bin/fix-rust-p2-check.sh

Current output looks like this:

$ fix-rust-p2-check.sh
+ cd /home/bhipple/src/nixpkgs
+ git checkout -f master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
+ git clean -ffdx
Removing new/
Removing old/
+ ATTRS=(broot fido2luks gitAndTools.git-workspace hexdino mdsh nix-du tre-command wagyu cargo exa nix-index ripgrep rustc rx termplay ja2-stracciatella parity tree-sitter)
+ mkdir -p old new
+ nix build -L -f . broot fido2luks gitAndTools.git-workspace hexdino mdsh nix-du tre-command wagyu cargo exa nix-index ripgrep rustc rx termplay ja2-stracciatella parity tree-sitter
+ mv result result-1 result-10 result-11 result-12 result-13 result-14 result-15 result-16 result-17 result-2 result-3 result-4 result-5 result-6 result-7 result-8 result-9 old/
+ fix-rust-p2.sh
Already on 'master'
Your branch is up to date with 'origin/master'.
+ nix build -L -f . broot fido2luks gitAndTools.git-workspace hexdino mdsh nix-du tre-command wagyu cargo exa nix-index ripgrep rustc rx termplay ja2-stracciatella parity tree-sitter
+ mv result result-1 result-10 result-11 result-12 result-13 result-14 result-15 result-16 result-17 result-2 result-3 result-4 result-5 result-6 result-7 result-8 result-9 new/
+ set +x
old
├── result -> /nix/store/isxbiasgzwalpjf4n4k92m2zc9lam67n-broot-0.13.1
├── result-1 -> /nix/store/jdyd1ira6sd4qrjqzgvngkz51zs5kfna-fido2luks-0.2.3
├── result-10 -> /nix/store/dan48igpc9zxplgc03057bdcjs5giby5-nix-index-0.1.2
├── result-11 -> /nix/store/5aj0ajqhwcfahgrz1nrflb39jr36gsj7-ripgrep-11.0.2
├── result-12 -> /nix/store/y41n9f36mhvsv6jr1fxfxgl781lhcz15-rustc-1.41.0
├── result-13 -> /nix/store/qs2gvhrhya62q9lcfsk2dgi8asc20s3j-rx-0.3.2
├── result-14 -> /nix/store/v8z8lb5f6mm6j2dksskncrdvrf3cdsi4-termplay-2.0.6
├── result-15 -> /nix/store/amjbisn3f0brb7g9gbks6zidq4f6irzl-ja2-stracciatella-0.16.1
├── result-16 -> /nix/store/7qn6ly7k9c0p269qckqzp6nf5767v0y0-parity-2.5.11
├── result-17 -> /nix/store/zdq97hi4fzcrqqfdcd1p93c12pjqkkfj-tree-sitter-0.15.7
├── result-2 -> /nix/store/d1apq07p5xc3qd3j7c1hx5vy6pqg9a5i-git-workspace-0.5.0
├── result-3 -> /nix/store/2zpwpqj60wjzlj1vggihp9snqzjy6jxv-hexdino-0.1.0
├── result-4 -> /nix/store/0j3igiwm5hd9yn2hmrzb7ck6s73vvx65-mdsh-0.3.0
├── result-5 -> /nix/store/51jfdd5rcy0qryxfhr5vavbfgvi6w50c-nix-du-0.3.1
├── result-6 -> /nix/store/4xrlbs7fpakm2yja3fg3wv1c3w9xqnaf-tre-0.2.2
├── result-7 -> /nix/store/lvcd0pkxnc274gqwaxn2z37hx8xfxg99-wagyu-0.6.1
├── result-8 -> /nix/store/7v2q1yrixn7qb7gbscxcqn2xw36mlaxy-cargo-1.41.0
└── result-9 -> /nix/store/icly3vax74dq60dmyn5879wpvqx5dd3l-exa-0.9.0
new
├── result -> /nix/store/isxbiasgzwalpjf4n4k92m2zc9lam67n-broot-0.13.1
├── result-1 -> /nix/store/jdyd1ira6sd4qrjqzgvngkz51zs5kfna-fido2luks-0.2.3
├── result-10 -> /nix/store/dan48igpc9zxplgc03057bdcjs5giby5-nix-index-0.1.2
├── result-11 -> /nix/store/5aj0ajqhwcfahgrz1nrflb39jr36gsj7-ripgrep-11.0.2
├── result-12 -> /nix/store/y41n9f36mhvsv6jr1fxfxgl781lhcz15-rustc-1.41.0
├── result-13 -> /nix/store/qs2gvhrhya62q9lcfsk2dgi8asc20s3j-rx-0.3.2
├── result-14 -> /nix/store/v8z8lb5f6mm6j2dksskncrdvrf3cdsi4-termplay-2.0.6
├── result-15 -> /nix/store/amjbisn3f0brb7g9gbks6zidq4f6irzl-ja2-stracciatella-0.16.1
├── result-16 -> /nix/store/7qn6ly7k9c0p269qckqzp6nf5767v0y0-parity-2.5.11
├── result-17 -> /nix/store/zdq97hi4fzcrqqfdcd1p93c12pjqkkfj-tree-sitter-0.15.7
├── result-2 -> /nix/store/d1apq07p5xc3qd3j7c1hx5vy6pqg9a5i-git-workspace-0.5.0
├── result-3 -> /nix/store/2zpwpqj60wjzlj1vggihp9snqzjy6jxv-hexdino-0.1.0
├── result-4 -> /nix/store/0j3igiwm5hd9yn2hmrzb7ck6s73vvx65-mdsh-0.3.0
├── result-5 -> /nix/store/51jfdd5rcy0qryxfhr5vavbfgvi6w50c-nix-du-0.3.1
├── result-6 -> /nix/store/4xrlbs7fpakm2yja3fg3wv1c3w9xqnaf-tre-0.2.2
├── result-7 -> /nix/store/lvcd0pkxnc274gqwaxn2z37hx8xfxg99-wagyu-0.6.1
├── result-8 -> /nix/store/7v2q1yrixn7qb7gbscxcqn2xw36mlaxy-cargo-1.41.0
└── result-9 -> /nix/store/icly3vax74dq60dmyn5879wpvqx5dd3l-exa-0.9.0

36 directories, 0 files
********************************************************************************
Diff
No hash diff generated. Success!

I've reviewed the diff as well and the rest of them look cookie-cutter, but if there's any one you're suspicious of let me know and I will verify that this PR does not change its hash or cause any rebuilds.

@bhipple

This comment has been minimized.

Copy link
Contributor Author

bhipple commented Feb 13, 2020

@GrahamcOfBorg build ripgrep nix-du

@GrahamcOfBorg eval

@@ -17,7 +17,7 @@
# Please set to true on any Rust package updates. Once all packages set this
# to true, we will delete and make it the default. For details, see the Rust
# section on the manual and ./README.md.
, legacyCargoFetcher ? true
, legacyCargoFetcher ? false

This comment has been minimized.

Copy link
@bhipple

bhipple Feb 13, 2020

Author Contributor

This is the actual change on the PR. Once merged I'll send another one updating the manual and deleting this doc comment.

Since the PR is so big and likely to generate merge conflicts as others submit PRs bumping Rust applications, for now I've just stuck to a diff that can be programmatically regenerated after a hard reset to origin/master, though I could make the sed smarter.

@bhipple

This comment has been minimized.

Copy link
Contributor Author

bhipple commented Feb 13, 2020

Nice to see ofborg agrees this causes 0 rebuilds :)

CC @worldofpeace @disassembler once merged I'll backport this to 20.03, but doing so requires #79981 first (which has the implementation, defaulted to false, and the removeAttrs fixes to allow this PR to be a clean no-rebuild change).

@worldofpeace

This comment has been minimized.

Copy link
Member

worldofpeace commented Feb 13, 2020

Nice to see ofborg agrees this causes 0 rebuilds :)

CC @worldofpeace @disassembler once merged I'll backport this to 20.03, but doing so requires #79981 first (which has the implementation, defaulted to false, and the removeAttrs fixes to allow this PR to be a clean no-rebuild change).

I approved it, but I will defer integration to those who are informed.

Copy link
Member

zimbatm left a comment

nice!

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 13, 2020

there are several merge conflicts, it would probably be prudent to get this in ASAP

@worldofpeace

This comment has been minimized.

Copy link
Member

worldofpeace commented Feb 13, 2020

there are several merge conflicts, it would probably be prudent to get this in ASAP

I believe it was mentioned the diff can be programmatically regenerated #79978 (comment)

@bhipple bhipple force-pushed the bhipple:feature/rust-migration-p2 branch from c6c2fc4 to db33314 Feb 14, 2020
@bhipple

This comment has been minimized.

Copy link
Contributor Author

bhipple commented Feb 14, 2020

Did a git fetch; git reset --hard origin/master, re-applied the diff generator, and pushed to PR. As before, no manual changes and no expected hash changes/rebuilds.

I also re-ran the fix-rust-p2-check.sh script posted above and verified there are still no rebuilds on the ~20 Rust packages I'm spot-checking.

Changes the default fetcher in the Rust Platform to be the newer
`fetchCargoTarball`, and changes every application using the current default to
instead opt out.

This commit does not change any hashes or cause any rebuilds. Once integrated,
we will start deleting the opt-outs and recomputing hashes.

See #79975 for details.
@bhipple bhipple force-pushed the bhipple:feature/rust-migration-p2 branch from db33314 to ade9c4b Feb 14, 2020
@bhipple

This comment has been minimized.

Copy link
Contributor Author

bhipple commented Feb 14, 2020

Went into conflict again; rebase and re-regenerated the diff again :)

Copy link
Contributor

jonringer left a comment

changes seems to be in anticipation to changes in the rust ecosystem.

LGTM

@jonringer jonringer merged commit eb11fea into NixOS:master Feb 14, 2020
12 checks passed
12 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.