Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[20.09] python3Packages.dask: limit processes on tests, fix downstream #100167

Merged
merged 9 commits into from Oct 12, 2020

Conversation

@jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 10, 2020

Motivation for this change

backport #99575

includes some build fixes, but also makes some package much faster to review (specifically tensorly)

also backported #95824 to fix tensorflow_2 build

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
jonringer added 5 commits Oct 4, 2020
(cherry picked from commit c65c94d)
(cherry picked from commit 7c841a2)
(cherry picked from commit b25c404)
(cherry picked from commit d65aba6)
(cherry picked from commit 74c997d)
raboof and others added 4 commits Aug 1, 2020
Also:
- patch to remove scipy requirement
- add cuda to RPATH
- don’t include nvidia_x11 (This isn’t needed, we can get it from
/run/opengl-driver being in the RPATH.)

Co-authored-by: Arnout Engelen <arnout@bzzt.net>
Co-authored-by: Daniël de Kok <me@github.danieldk.eu>
(cherry picked from commit 1e19d44)
also disable on python 2.7
Co-authored-by: Jon <jonringer@users.noreply.github.com>

(cherry picked from commit 9cd240c)
(cherry picked from commit 59eecac)
@jonringer
Copy link
Contributor Author

@jonringer jonringer commented Oct 12, 2020

No regressions: other packages are already failing on release-20.09. Overall improvement

https://github.com/NixOS/nixpkgs/pull/100167
4 packages marked as broken and skipped:
python37Packages.qasm2image python37Packages.rl-coach python37Packages.tensorflowWithCuda python38Packages.qasm2image

6 packages failed to build:
python37Packages.clifford python37Packages.graph_nets python37Packages.optuna python37Packages.pymc3 python37Packages.tensorflow-bin python37Packages.tensorflow-bin_2

75 packages built:
python27Packages.tensorflow-estimator_2 python37Packages.Keras python37Packages.annoy python37Packages.arviz python37Packages.awkward python37Packages.baselines python37Packages.bayespy python37Packages.caffe python37Packages.datashader python37Packages.dcmstack python37Packages.dicom2nifti python37Packages.dipy python37Packages.dm-sonnet python37Packages.edward python37Packages.h5netcdf python37Packages.h5py python37Packages.h5py-mpi python37Packages.heudiconv python37Packages.hickle python37Packages.intake python37Packages.keras-applications python37Packages.mask-rcnn python37Packages.nibabel python37Packages.nilearn python37Packages.nipy python37Packages.nipype python37Packages.nitime python37Packages.phonopy python37Packages.pybids python37Packages.pywick python37Packages.qiskit python37Packages.qiskit-aqua python37Packages.sumo python37Packages.tensorflow python37Packages.tensorflow_2 python37Packages.tensorflow-estimator_2 python37Packages.tensorflow-probability python37Packages.tensorly python37Packages.tflearn python37Packages.uproot python37Packages.uproot-methods python37Packages.worldengine python38Packages.Keras python38Packages.annoy python38Packages.awkward python38Packages.bayespy python38Packages.caffe python38Packages.clifford python38Packages.datashader python38Packages.dicom2nifti python38Packages.dipy python38Packages.h5netcdf python38Packages.h5py python38Packages.h5py-mpi python38Packages.hickle python38Packages.intake python38Packages.keras-applications python38Packages.mask-rcnn python38Packages.nibabel python38Packages.nilearn python38Packages.nipy python38Packages.nipype python38Packages.phonopy python38Packages.pybids python38Packages.pywick python38Packages.qiskit python38Packages.qiskit-aqua python38Packages.sumo python38Packages.tensorflow_2 python38Packages.tensorflow-estimator_2 python38Packages.tensorly python38Packages.uproot python38Packages.uproot-methods worldengine-cli visidata
@jonringer jonringer merged commit 6c2310c into NixOS:release-20.09 Oct 12, 2020
16 checks passed
16 checks passed
action
Details
Evaluation Performance Report Evaluator Performance Report
Details
Wait for ofborg
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="c0af2ab"; rev="c0af2ab94821afdc81bbf5763974abacbbdc91f6"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@jonringer jonringer deleted the jonringer:backport-python-dask branch Oct 12, 2020
@jonringer jonringer mentioned this pull request Oct 16, 2020
4 of 10 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.