Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildRustCrate: add host platform to rlib hash suffix #105305

Merged
merged 1 commit into from Nov 29, 2020

Conversation

@lopsided98
Copy link
Contributor

@lopsided98 lopsided98 commented Nov 29, 2020

Motivation for this change

If a crate depends on both the host and build versions of a crate when cross-compiling, both versions of the crate have the same rlib name, and they clobber each other in the deps directory. This PR adds the host platform target triple to the hash, so the crates end up with unique hashes.

Is there a reason ___ is used to separate the components?

cc @Ericson2314 @andir

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Nov 29, 2020

Is there a reason ___ is used to separate the components?

I bet someone was being cautious about pre-image attacks. Not sure if this is necessary or even sufficient, but I suppose we can leave it for now. Maybe if you want and a TODO asking someone to explain.

@lopsided98 lopsided98 force-pushed the build-rust-crate-platform-hash branch from 096c213 to f46a70c Nov 29, 2020
@lopsided98
Copy link
Contributor Author

@lopsided98 lopsided98 commented Nov 29, 2020

This breaks buildRustCrateTests because they check the specific hash value. We should probably just allow any hash, but that will require rewriting the test infrastructure a bit.

@lopsided98 lopsided98 force-pushed the build-rust-crate-platform-hash branch from f46a70c to 8c47905 Nov 29, 2020
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Nov 29, 2020

Yeah I would just focus on testing that things that shouldn't be equal indeed are. Nix makes it very easy to make it deterministic, but leaves it to the other to make the equivalence class sufficiently fine-grained.

@Ericson2314 Ericson2314 merged commit 7781642 into NixOS:master Nov 29, 2020
17 checks passed
17 checks passed
@github-actions[bot]
tests
Details
@github-actions[bot]
action
Details
@ofborg[bot]
Evaluation Performance Report Evaluator Performance Report
Details
@github-actions[bot]
Wait for ofborg
Details
@ofborg[bot]
grahamcofborg-eval ^.^!
Details
@ofborg[bot]
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
@ofborg[bot]
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
@ofborg[bot]
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./pkgs/t
Details
@ofborg[bot]
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
@ofborg[bot]
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./nixos/
Details
@ofborg[bot]
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./nixos/
Details
@ofborg[bot]
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./nixos/
Details
@ofborg[bot]
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./pkgs/t
Details
@ofborg[bot]
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./pkgs/t
Details
@ofborg[bot]
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="8c47905"; rev="8c479059b9a1a0010deaa2b2d815789febb030a1"; } ./pkgs/t
Details
@ofborg[bot]
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
@ofborg[bot]
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Nov 29, 2020

Oh I forgot about the tests, I guess I'll go fix myself.

Ericson2314 pushed a commit that referenced this issue Nov 29, 2020
As @lopsided98 points out in #105305, we're probably better off not
asserting exact hashes, but that is a more involved fix.
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Nov 29, 2020

Fixed tests in #105379.

@lopsided98 lopsided98 deleted the build-rust-crate-platform-hash branch Nov 29, 2020
@andir
Copy link
Member

@andir andir commented Nov 30, 2020

Ugh, the commit message doesn't cary any of the detail required to judge the change. We only have that in this PR.. This will make it harder to figure out why we did something. In my initial refactoring the git history was the most valuable asset we had on this :/

Regardless of that I think this is a good change. Thanks @lopsided98!

Ericson2314 pushed a commit that referenced this issue Dec 19, 2020
As @lopsided98 points out in #105305, since the hashes are now target
sensative, and until we find reason to actually care to test what they
are exactly, we are best just normalizing them away in the tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants