Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python packages set: major updates #105368

Merged
merged 398 commits into from Dec 1, 2020
Merged

Python packages set: major updates #105368

merged 398 commits into from Dec 1, 2020

Conversation

@FRidh
Copy link
Member

@FRidh FRidh commented Nov 29, 2020

Motivation for this change

It's been a long time since we did major updates in the whole set. Applications were not automatically updated, but likely need some fixing as well.

https://hydra.nixos.org/jobset/nixpkgs/python-unstable

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@FRidh
Copy link
Member Author

@FRidh FRidh commented Nov 29, 2020

numpy needs numpy/numpy#17776

@jonringer
Copy link
Contributor

@jonringer jonringer commented Nov 29, 2020

the use of flit_core is making gimp fail because python2Packages.flit_core is disabled, which failing the eval as gimp is a darwin constituent https://gist.github.com/GrahamcOfBorg/f0aa782e418d7d426a7bf7a5a7372186

@jonringer
Copy link
Contributor

@jonringer jonringer commented Nov 29, 2020

maybe we should revert the packaging bump until they figure out their existence.

@FRidh
Copy link
Member Author

@FRidh FRidh commented Nov 30, 2020

the use of flit_core is making gimp fail because python2Packages.flit_core is disabled, which failing the eval as gimp is a darwin constituent https://gist.github.com/GrahamcOfBorg/f0aa782e418d7d426a7bf7a5a7372186

In principle the Python 3 flit can make a wheel for Python 2 (they're universal wheels). I've been avoiding this but can use this fact.

Easiest here is to just keep another package for Python 2.

@FRidh
Copy link
Member Author

@FRidh FRidh commented Nov 30, 2020

In principle the Python 3 flit can make a wheel for Python 2 (they're universal wheels). I've been avoiding this but can use this fact.

I had a quick look at this approach. Fun stuff! So, in principle we could run the flit command, Python 3 version, to build the wheel. However, e.g. packaging uses PEP 517, so pip will attempt to use that method to find the backend. Then, it fails, because it cannot find the Python 3 version. Sigh.

Really the best approach here, if we were caring sufficiently for Python 2, would be to separate the wheel building part into a separate derivation. The installation function could then simply pick the universal wheel built with Python 3. Now I would still like to do that some day, and not for Python 2, but for other reasons, but not anytime soon.

So for now I suggest we keep the older packaging for Python 2 around.

@FRidh FRidh force-pushed the python-unstable branch from b31f6e7 to b0e3728 Dec 1, 2020
@FRidh FRidh merged commit 9750653 into staging Dec 1, 2020
2 of 4 checks passed
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Dec 1, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/upcoming-python-breakage/7554/19

@FRidh
Copy link
Member Author

@FRidh FRidh commented Dec 1, 2020

Thank you @jonringer. There are about 900 failures left, which is about the same as it was before. I've updated to the latest pip.

@FRidh FRidh mentioned this pull request Dec 1, 2020
10 tasks
@TredwellGit
Copy link
Member

@TredwellGit TredwellGit commented Dec 3, 2020

After this was merged on staging:

$ nixos-rebuild build -I nixpkgs=.
building Nix...
building the system configuration...
error: pytest-xdist-2.1.0 not supported for interpreter python2.7
(use '--show-trace' to show detailed location information)

@TredwellGit
Copy link
Member

@TredwellGit TredwellGit commented Dec 3, 2020

Seems to be caused by TeX Live depending on Python 2.

@TredwellGit
Copy link
Member

@TredwellGit TredwellGit commented Dec 3, 2020

I have TeX Live working; will request pull later.

teto
Copy link
Contributor

teto commented on 04a0d45 Dec 20, 2020

the dependencies have been changed, it's now using aiobotocore. I have local changes I will submit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants