Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_INSTALL_TMPDIR #91272

Merged
merged 1 commit into from Jul 2, 2020

Conversation

@misuzu
Copy link
Contributor

misuzu commented Jun 22, 2020

Motivation for this change

Another approach to #81441 as #90208 caused too many regressions.

With this patch pip will use NIX_PIP_INSTALL_TMPDIR as temporary directory when defined, usual code path will be used otherwise.

I've built some packages to make sure it's working:

  • pkgs.python2Packages.flask
  • pkgs.python2Packages.invoke
  • pkgs.python3Packages.flask
  • pkgs.python3Packages.invoke
  • pkgs.python3Packages.aiohttp - not reproducible (most likely unrelated)
  • pkgs.gtk-doc
  • nixos.iso_minimal.x86_64-linux - not reproducible (obvious)
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@FRidh
Copy link
Member

FRidh commented Jun 22, 2020

This way pip will always listen to the TMPDIR var, also when not used within a Nix build, and also for other operations.

Maybe following is a better place https://github.com/pypa/pip/blob/642fb07337a084252ae43c6b596dc18e434b2807/src/pip/_internal/operations/install/wheel.py#L676. Something like _temp_dir_for_testing = _temp_dir_for_testing or _os.environ.get("NIX_PIP_TMPDIR").

@misuzu
Copy link
Contributor Author

misuzu commented Jun 22, 2020

Maybe following is a better place

Yeah, less code to maintain, i like it.
I'll check it now.

@misuzu misuzu force-pushed the misuzu:pip-reproducible branch from 3dc0b91 to be07c60 Jun 22, 2020
@misuzu misuzu changed the title python3Packages.pip: allow setting reproducible temporary directory via TMPDIR python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_TMPDIR Jun 22, 2020
@misuzu misuzu changed the title python3Packages.pip: allow setting reproducible temporary directory via TMPDIR python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_TMPDIR Jun 22, 2020
@FRidh
Copy link
Member

FRidh commented Jun 22, 2020

even more specific name NIX_PIP_INSTALL_TMPDIR. This one really should be just internal.

@misuzu misuzu force-pushed the misuzu:pip-reproducible branch from be07c60 to b54fbd2 Jun 22, 2020
…ia NIX_PIP_INSTALL_TMPDIR
@misuzu misuzu force-pushed the misuzu:pip-reproducible branch from b54fbd2 to a451fdb Jun 22, 2020
@misuzu misuzu changed the title python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_TMPDIR python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_INSTALL_TMPDIR Jun 22, 2020
@misuzu
Copy link
Contributor Author

misuzu commented Jun 22, 2020

It seems to work as expected too, both python2 and python3.

@misuzu misuzu requested a review from FRidh Jun 26, 2020
@FRidh FRidh self-assigned this Jun 27, 2020
@FRidh FRidh merged commit aedbade into NixOS:staging Jul 2, 2020
14 checks passed
14 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="a451fdb"; rev="a451fdbacaafac5396425a6b8880bdfc492ab6cf"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.