Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[20.09] pythonPackages.pymc3: fix build, init pythonPackages.fastprogress at 1.0.0 #100955

Merged

Conversation

@risicle
Copy link
Contributor

@risicle risicle commented Oct 18, 2020

Motivation for this change

ZHF: #97479

This is a modified & combined backport of #99587 and #100867.

Because 20.09 is missing #97597 and #93560 (and I think including them at this stage might be a bit risky), importing/using theano from within a nix build environment (i.e. with NIX_ENFORCE_PURITY=1) will only succeed if the tmpdir used is actually literally /tmp, which you can't really do from a sandboxed build. So this uses libredirect to convince the compiler wrapper we are actually working with /tmp.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@risicle risicle requested review from FRidh and jonringer as code owners Oct 18, 2020
@risicle
Copy link
Contributor Author

@risicle risicle commented Oct 18, 2020

@ofborg build python37Packages.pymc3

Copy link
Member

@das-g das-g left a comment

Please use correct grammar in comments.

pkgs/development/python-modules/Theano/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/pymc3/default.nix Outdated Show resolved Hide resolved
risicle added 3 commits Oct 17, 2020
(modified cherry pick of commit 4a9dba6)

some tricks are required to make this work because #93560 and #97597
didn't make it into this branch, but this example shows what is required
to import theano from within a nix build environment on this branch.
(modified cherry-pick of commit 3c3295a)
@risicle risicle force-pushed the risicle:ris-theano-tmpdir-redirect-fix-r20.09 branch from 264840d to f447235 Oct 18, 2020
@risicle
Copy link
Contributor Author

@risicle risicle commented Oct 18, 2020

Er.. not sure if you're serious, but ok...

@das-g
Copy link
Member

@das-g das-g commented Oct 18, 2020

Result of nixpkgs-review pr 100955 1

10 packages built:
  • python27Packages.Lasagne
  • python27Packages.Theano (python27Packages.TheanoWithoutCuda)
  • python27Packages.TheanoWithCuda
  • python37Packages.Theano (python37Packages.TheanoWithoutCuda)
  • python37Packages.TheanoWithCuda
  • python37Packages.fastprogress
  • python37Packages.pymc3
  • python38Packages.Theano (python38Packages.TheanoWithoutCuda)
  • python38Packages.TheanoWithCuda
  • python38Packages.fastprogress

@das-g
das-g approved these changes Oct 19, 2020
Copy link
Contributor

@jonringer jonringer left a comment

LGTM

Result of nixpkgs-review pr 100955 1

10 packages built:
  • python27Packages.Lasagne
  • python27Packages.Theano (python27Packages.TheanoWithoutCuda)
  • python27Packages.TheanoWithCuda
  • python37Packages.Theano (python37Packages.TheanoWithoutCuda)
  • python37Packages.TheanoWithCuda
  • python37Packages.fastprogress
  • python37Packages.pymc3
  • python38Packages.Theano (python38Packages.TheanoWithoutCuda)
  • python38Packages.TheanoWithCuda
  • python38Packages.fastprogress

@jonringer jonringer merged commit 4fc2635 into NixOS:release-20.09 Oct 19, 2020
16 checks passed
16 checks passed
@github-actions
action
Details
@ofborg
Evaluation Performance Report Evaluator Performance Report
Details
@github-actions
Wait for ofborg
Details
@ofborg
grahamcofborg-eval ^.^!
Details
@ofborg
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
@ofborg
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
@ofborg
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./pkgs/t
Details
@ofborg
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
@ofborg
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./nixos/
Details
@ofborg
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./nixos/
Details
@ofborg
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./nixos/
Details
@ofborg
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./pkgs/t
Details
@ofborg
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./pkgs/t
Details
@ofborg
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="f447235"; rev="f447235c9718ea70c5d65b8918ec2e6101a3db37"; } ./pkgs/t
Details
@ofborg
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
@ofborg
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@twhitehead
Copy link
Contributor

@twhitehead twhitehead commented Oct 22, 2020

@risicle BTW, just wanted to say thanks for working this all out. Have been reading through the various threads and can see this was quite a mess for you to straighten out. 👍

@risicle
Copy link
Contributor Author

@risicle risicle commented Oct 22, 2020

No problem - you got the real fix in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants