Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backport fixes to Dask and Distributed to release-18.09 #54974

Merged
merged 6 commits into from Feb 1, 2019
Merged

Backport fixes to Dask and Distributed to release-18.09 #54974

merged 6 commits into from Feb 1, 2019

Conversation

CMCDragonkai
Copy link
Member

@CMCDragonkai CMCDragonkai commented Jan 31, 2019 •
edited

Motivation for this change

Fixes #54875

This PR fixes import dask.distributed.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@CMCDragonkai CMCDragonkai force-pushed the release-18.09 branch 2 times, most recently from 350851e to 582c7e2 Compare January 31, 2019 04:00
@CMCDragonkai
Copy link
Member Author

CMCDragonkai commented Jan 31, 2019 •
edited

Tested that the import works:

nix-shell -I nixpkgs=$(pwd) -p "python3.withPackages (ps: with ps; [ dask distributed ])" --run "python -c 'import dask.distributed'"

@CMCDragonkai CMCDragonkai changed the title WIP: Backport fixes to Dask and Distributed to release-18.09 Backport fixes to Dask and Distributed to release-18.09 Jan 31, 2019
@CMCDragonkai CMCDragonkai mentioned this pull request Jan 31, 2019
Closed
@kalbasit kalbasit self-requested a review January 31, 2019 04:48
kalbasit
kalbasit requested changes Jan 31, 2019
View reviewed changes
pkgs/development/python-modules/pytest-repeat/default.nix

buildPythonPackage rec {
pname = "pytest-repeat";
version = "0.6.0";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly for this package.

pkgs/development/python-modules/distributed/default.nix Outdated

buildPythonPackage rec {
pname = "distributed";
version = "1.22.1";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a newer version per the one on master.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you point out the specific commits to use? Otherwise the purpose of this is to solve the bug, not update to latest.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're already bumping from 1.15.1 to 1.22.1. Sure, it's 816cc6d.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could be a separate PR?

@dotlambda
Copy link
Member

You must use git cherry-pick -x.

@dotlambda
Copy link
Member

I don't see a reason to backport more commits than necessary for fixing the bug.

@costrouc @CMCDragonkai
pythonPackages.pytest-faulthandler: init at 1.5.0
ff5a9b9 
 - compatible with 2.7 and 3+
 - all test pass.

(cherry picked from commit 5704642)
@costrouc @CMCDragonkai
pythonPackages.pytest-repeat: init at 0.6.0
fcd7df5 
 - compatible with 2.7, 3+
 - all tests pass and running

(cherry picked from commit 7b92017)
@costrouc @CMCDragonkai
pythonPackages.distributed: 1.15.1 -> 1.22.1
19a3d55 
The github repository was downloaded instead of the pypi repository
for testing (needed `conftest.py`). Major work was done on the
underlying dependencies to make distributed work on python 2.7,
3+. Note that the test **do** take a significant amount of time (10-15
minutes).

 - moved to `python-modules`
 - compatible with 2.7, 3+
 - all tests pass (previously tests were not run)

(cherry picked from commit 4f70170)
@costrouc @CMCDragonkai
fixup! merge: fixing changes with nixpkgs since pull-request
802e8d2 
(cherry picked from commit 027b9be)
@CMCDragonkai
Copy link
Member Author

I've used the -x option now.

costrouc and others added 2 commits February 1, 2019 12:52
@costrouc @CMCDragonkai
pythonPackages.distributed: refactor move to fetchPypi
283985b 
There is a bug when using git repo in that the package does not get a
version (shows up as `0+unknown`). Using pypi fixes this issue allows
allows for auto upgrades.

(cherry picked from commit e5cd420)
@FRidh @CMCDragonkai
python: distributed: 1.23.1 -> 1.23.3
5272775 
(cherry picked from commit e8783b8)
@CMCDragonkai
Copy link
Member Author

CMCDragonkai commented Feb 1, 2019 •
edited

@kalbasit I've added 2 more commits of distributed until the date that dask got updated as well. These 2 commits were additional to just distributed. I think that's safe, unless I introduce more bugs due to version incompatibilities.

Basically I stopped at 1st Nov 2018. That goes for pytest-repeat as well.

https://github.com/NixOS/nixpkgs/commits/816cc6dc8384af52f050090be2529c7ffaf44145/pkgs/development/python-modules/distributed/default.nix

https://github.com/NixOS/nixpkgs/commits/1a51abe9c12d1093527fec48edaac1e6f61f04d7/pkgs/development/python-modules/dask/default.nix

@CMCDragonkai
Copy link
Member Author

@dotlambda I think this should be ready.

@dotlambda dotlambda merged commit 2481296 into NixOS:release-18.09 Feb 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kalbasit kalbasit kalbasit requested changes

@FRidh FRidh Awaiting requested review from FRidh FRidh is a code owner

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@CMCDragonkai @dotlambda @kalbasit @GrahamcOfBorg @costrouc @FRidh