Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildRustCrate: Replace hyphen with underscore in env variables #88054

Merged
merged 2 commits into from May 26, 2020

Conversation

@notriddle
Copy link
Contributor

notriddle commented May 18, 2020

This fixes a bug that prevents encoding_c from building.

Motivation for this change

I tried to use crate2nix to build a Rust package that depends on mozjs. This depends on encoding_c, which passes this output from build.rs:

cargo:include-dir=$PWD/includes

Which gets rewritten to this env file:

export DEPS_encoding_c_INCLUDE-DIR=$PWD/includes

This is invalid; nix doesn't allow environment variables to contain hyphens. In any case, it also doesn't match up with how mozjs is trying to reference it here. The mozjs build script expects it to be spelled DEPS_ENCODING_C_INCLUDE_DIR, with all of it in SCREAMING_CASE.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@andir
Copy link
Member

andir commented May 20, 2020

@ofborg build buildRustCrateTests

@andir andir self-assigned this May 20, 2020
Copy link
Member

andir left a comment

Thank you! The change looks good. Do you mind adding a test for this edge case? It will prevent us from regressing on this issue with the next refactoring.

@Mic92 Mic92 changed the title Replace hyphen with underscore in env variables buildRustCrate: Replace hyphen with underscore in env variables May 20, 2020
@notriddle
Copy link
Contributor Author

notriddle commented May 24, 2020

I couldn't find any good docs on this, so just as a note, I actually ran that test suite by executing nix-build -A buildRustCrateTests.test from the root of the nixpkgs repository. Is there a better way?

Running it against master, with the test case cherry-picked
$ nix-build -A buildRustCrateTests.test
these derivations will be built:
  /nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv
  /nix/store/dnjnc55hzx1a9qqnnxx4j8h6kqsw16qc-run-buildRustCrate-foo-test.drv
  /nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv
building '/nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/ccsm2m5ihgngmdprix638dnvyk7njmqb-build-script-and-include-dir-foo
source root is build-script-and-include-dir-foo
patching sources
configuring
Running cd .
/nix/store/969yral7dpsanjracak90xy0nbny7jpb-rust_bar-0.1.0-lib/env: line 1: export: `DEP_bar_INCLUDE-DIR=/build/build-script-and-include-dir-bar/src': not a valid identifier
builder for '/nix/store/qrx5z7fzygckbk3dqvif0kczylk2m6ya-rust_foo-0.1.0.drv' failed with exit code 1
cannot build derivation '/nix/store/dnjnc55hzx1a9qqnnxx4j8h6kqsw16qc-run-buildRustCrate-foo-test.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv': 1 dependencies couldn't be built
error: build of '/nix/store/qzqv6gipvp0zjwpbivz2hj7spnjib9rw-buildRustCrate-tests.drv' failed
Running against my patch branch
$ nix-build -A buildRustCrateTests.test
these derivations will be built:
  /nix/store/fkc5r6ff6h610swk8psrvab21ycvqk8d-src.drv
  /nix/store/vigvvmh0iqfmrbv7dz3b4xp6d7sjjg5g-build-script-and-include-dir-bar.drv
  /nix/store/587pnaryla3d990drm0r4wwsxjv1drnd-rust_bar-0.1.0.drv
  /nix/store/m6657w9hp7xq7k8288y45ry62knkqn1p-rust_foo-0.1.0.drv
  /nix/store/hkc6h7rl07zz2sagvlk1kinhp5x2k4j7-run-buildRustCrate-foo-test.drv
  /nix/store/f8x6khaaqgqf75kjzb5r3qzdcjwnf5b6-buildRustCrate-tests.drv
building '/nix/store/fkc5r6ff6h610swk8psrvab21ycvqk8d-src.drv'...
building '/nix/store/vigvvmh0iqfmrbv7dz3b4xp6d7sjjg5g-build-script-and-include-dir-bar.drv'...
building '/nix/store/587pnaryla3d990drm0r4wwsxjv1drnd-rust_bar-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/iqq9iwprbg5dx6fxcw1j405v792vgzr5-build-script-and-include-dir-bar
source root is build-script-and-include-dir-bar
patching sources
configuring
Running cd .
Building build.rs (bar)
Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --out-dir target/build/bar --emit=dep-info,link -L dependency=target/buildDeps --cap-lints allow --color always
building
Building src/lib.rs (bar)
Running rustc --crate-name bar src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ -C metadata=558120c4eb -C extra-filename=-558120c4eb --crate-type lib -L /build/build-script-and-include-dir-bar/target/build/bar.out --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
patching script interpreter paths in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0
checking for references to /build/ in /nix/store/6fn30vzgln2ypbsa2s1pr7w9shbqz8g9-rust_bar-0.1.0...
shrinking RPATHs of ELF executables and libraries in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib
shrinking /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/bar/build_script_build
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib 
patching script interpreter paths in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib
checking for references to /build/ in /nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib...
building '/nix/store/m6657w9hp7xq7k8288y45ry62knkqn1p-rust_foo-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/ccsm2m5ihgngmdprix638dnvyk7njmqb-build-script-and-include-dir-foo
source root is build-script-and-include-dir-foo
patching sources
configuring
Running cd .
Building build.rs (foo)
Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --out-dir target/build/foo --emit=dep-info,link -L dependency=target/buildDeps --extern bar=/nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/libbar-558120c4eb.rlib --cap-lints allow --color always
building
Building foo (src/main.rs)
Running rustc --crate-name foo src/main.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern bar=/nix/store/gdfh656q4wk2cl54j2rq4ls6dfkc08zs-rust_bar-0.1.0-lib/lib/libbar-558120c4eb.rlib --out-dir target/bin -L dependency=target/deps --cap-lints allow -L /build/build-script-and-include-dir-foo/target/build/foo.out --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0
shrinking /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0/bin/foo
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0/bin 
patching script interpreter paths in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0
checking for references to /build/ in /nix/store/ndi763vpajklcxs1wy7dhmvpsk2fds8b-rust_foo-0.1.0...
shrinking RPATHs of ELF executables and libraries in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib
shrinking /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib/lib/foo/build_script_build
strip is /nix/store/a57856fs4m8ir6vlv14h3gq3sv9aq2lb-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib/lib 
patching script interpreter paths in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib
checking for references to /build/ in /nix/store/fbifzwkwh6vxc4ghjs419p6xwwdx81mv-rust_foo-0.1.0-lib...
building '/nix/store/hkc6h7rl07zz2sagvlk1kinhp5x2k4j7-run-buildRustCrate-foo-test.drv'...
building '/nix/store/f8x6khaaqgqf75kjzb5r3qzdcjwnf5b6-buildRustCrate-tests.drv'...
/nix/store/1rqydxhx6lmscia5vslmi6h299c778ax-buildRustCrate-tests

In any case, @andir, I've added a regression test.

notriddle added 2 commits May 24, 2020
This fixes a bug that prevents encoding_c from building.
@notriddle notriddle force-pushed the notriddle:patch-1 branch from ae2756c to 880d86c May 24, 2020
@ofborg ofborg bot requested a review from andir May 24, 2020
@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

@ofborg build buildRustCrateTests

@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

Looks like all the tests are passing, and there is now a regression test… let's merge :)

@Ekleog Ekleog merged commit c21cbf2 into NixOS:master May 26, 2020
17 checks passed
17 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
buildRustCrateTests on aarch64-linux Success
Details
buildRustCrateTests on x86_64-darwin Success
Details
buildRustCrateTests on x86_64-linux Success
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="880d86c"; rev="880d86c25094294f0523f984ffa742ff460c261c"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@Ekleog
Copy link
Member

Ekleog commented May 26, 2020

@andir Do you think this should be backported? I'm a bit cautious about it, seeing as it's infrastructure, but on the other hand as it's a quite significant bug…?

@andir
Copy link
Member

andir commented May 26, 2020

Ekleog added a commit that referenced this pull request May 30, 2020
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.

(cherry picked from commit c21cbf2)
@Ekleog
Copy link
Member

Ekleog commented May 30, 2020

Done :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.