Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python fix 9 broken packages #65023

Closed
wants to merge 8 commits into from
Closed

Conversation

@costrouc
Copy link
Contributor

costrouc commented Jul 18, 2019

Motivation for this change

Closes PR #65003 and adds the packages that were merged into python-unstable into master. I will merge into staging if this is too large of a change

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

cc: @FRidh

costrouc added 7 commits Jul 18, 2019
@costrouc costrouc requested a review from FRidh as a code owner Jul 18, 2019
@costrouc
Copy link
Contributor Author

costrouc commented Jul 18, 2019

Running nix-review right now

@ofborg ofborg bot added the 6.topic: python label Jul 18, 2019
@costrouc
Copy link
Contributor Author

costrouc commented Jul 18, 2019

Results

Result of nix-review pr 65023 1

61 package failed to build:
  • aws-sam-cli (docker_compose)
  • aws_shell (docker_compose)
  • awscli (docker_compose)
  • buildbot (docker_compose)
  • csvs-to-sqlite (docker_compose)
  • elasticsearch-curator (docker_compose)
  • fedpkg (docker_compose)
  • flexget (docker_compose)
  • iceshelf (docker_compose)
  • nbstripout (docker_compose)
  • nixops_1_6_1 (docker_compose)
  • poretools (docker_compose)
  • pyCA (docker_compose)
  • pyload (docker_compose)
  • python27Packages.altair (docker_compose)
  • python27Packages.atomman (docker_compose)
  • python27Packages.awkward (docker_compose)
  • python27Packages.bkcharts (docker_compose)
  • python27Packages.blaze (docker_compose)
  • python27Packages.blivet (docker_compose)
  • python27Packages.bokeh (docker_compose)
  • python27Packages.bugwarrior (docker_compose)
  • python27Packages.celery (docker_compose)
  • python27Packages.cfn-lint (docker_compose)
  • python27Packages.csvs-to-sqlite (docker_compose)
  • python27Packages.cufflinks (docker_compose)
  • python27Packages.dask (docker_compose)
  • python27Packages.dask-glm (docker_compose)
  • python27Packages.dask-image (docker_compose)
  • python27Packages.dask-jobqueue (docker_compose)
  • python27Packages.dask-ml (docker_compose)
  • python27Packages.dask-mpi (docker_compose)
  • python27Packages.distributed (docker_compose)
  • python27Packages.djmail (docker_compose)
  • python27Packages.dm-sonnet (docker_compose)
  • python27Packages.drms (docker_compose)
  • python27Packages.fastparquet (docker_compose)
  • python27Packages.geopandas (docker_compose)
  • python27Packages.glymur (docker_compose)
  • python27Packages.google_cloud_bigquery (docker_compose)
  • python27Packages.google_cloud_monitoring (docker_compose)
  • python27Packages.graph_nets (docker_compose)
  • python27Packages.histbook (docker_compose)
  • python27Packages.holoviews (docker_compose)
  • python27Packages.hvplot (docker_compose)
  • python27Packages.image-match (docker_compose)
  • python27Packages.imbalanced-learn (docker_compose)
  • python27Packages.koji (docker_compose)
  • python27Packages.mapsplotlib (docker_compose)
  • python27Packages.moto (docker_compose)
  • python27Packages.nixpart0 (docker_compose)
  • python27Packages.odo (docker_compose)
  • python27Packages.osc (docker_compose)
  • python27Packages.pandas (docker_compose)
  • python27Packages.panel (docker_compose)
  • python27Packages.paperspace (docker_compose)
  • python27Packages.partd (docker_compose)
  • python27Packages.pims (docker_compose)
  • python27Packages.pvlib (docker_compose)
  • python27Packages.pyarrow (docker_compose)
  • python27Packages.pycurl (docker_compose)
51 package were build:
  • aws-google-auth (docker_compose)
  • awslogs (docker_compose)
  • caffeine-ng (docker_compose)
  • credstash (docker_compose)
  • devpi-server (docker_compose)
  • docker-compose (docker_compose)
  • luigi (docker_compose)
  • matrix-synapse (docker_compose)
  • nixops (docker_compose)
  • nixopsUnstable (docker_compose)
  • or-tools (docker_compose)
  • poetry (docker_compose)
  • python27Packages.ansible-kernel (docker_compose)
  • python27Packages.argon2_cffi (docker_compose)
  • python27Packages.astunparse (docker_compose)
  • python27Packages.aws-adfs (docker_compose)
  • python27Packages.aws-sam-translator (docker_compose)
  • python27Packages.baselines (docker_compose)
  • python27Packages.boto3 (docker_compose)
  • python27Packages.cartopy (docker_compose)
  • python27Packages.chalice (docker_compose)
  • python27Packages.colorcet (docker_compose)
  • python27Packages.credstash (docker_compose)
  • python27Packages.cupy (docker_compose)
  • python27Packages.edward (docker_compose)
  • python27Packages.fiona (docker_compose)
  • python27Packages.flask-restplus (docker_compose)
  • python27Packages.flowlogs_reader (docker_compose)
  • python27Packages.fs-s3fs (docker_compose)
  • python27Packages.gast (docker_compose)
  • python27Packages.gensim (docker_compose)
  • python27Packages.glob2 (docker_compose)
  • python27Packages.ipywidgets (docker_compose)
  • python27Packages.jsonschema (docker_compose)
  • python27Packages.jupyter (docker_compose)
  • python27Packages.jupytext (docker_compose)
  • python27Packages.kmapper (docker_compose)
  • python27Packages.modeled (docker_compose)
  • python27Packages.moretools (docker_compose)
  • python27Packages.nbconvert (docker_compose)
  • python27Packages.nbformat (docker_compose)
  • python27Packages.nbmerge (docker_compose)
  • python27Packages.nbsmoke (docker_compose)
  • python27Packages.nbsphinx (docker_compose)
  • python27Packages.nbval (docker_compose)
  • python27Packages.notebook (docker_compose)
  • python27Packages.ortools (docker_compose)
  • python27Packages.passlib (docker_compose)
  • python27Packages.plotly (docker_compose)
  • python27Packages.poetry (docker_compose)
  • python37Packages.ortools (docker_compose)
@costrouc
Copy link
Contributor Author

costrouc commented Jul 18, 2019

Does this mean that docker_compose is broken right now?

@costrouc costrouc force-pushed the costrouc:python-fix-broken branch from c3120da to 65ea1cc Jul 18, 2019
@ofborg ofborg bot requested a review from domenkozar Jul 18, 2019
@jonringer
Copy link
Contributor

jonringer commented Jul 18, 2019

it looks like your jsonschema bump to the next major version is what broke most things.

The most proper (but pain-in-the-***) fix I can think of, would be to pin pythonPackages.jsonschema_2 and go to each package that still requires it and amend them.

Collecting jsonschema~=2.6 (from cfn-lint==0.19.1)
  ERROR: Could not find a version that satisfies the requirement jsonschema~=2.6 (from cfn-lint==0.19.1) (from versions: none)
ERROR: No matching distribution found for jsonschema~=2.6 (from cfn-lint==0.19.1)
builder for '/nix/store/aav481gj38kdy5qj5rzm2qrjx7gsvlsk-python3.7-cfn-lint-0.19.1.drv' failed with exit code 1
cannot build derivation '/nix/store/v1rdmnwwf1q04jggbp1r53nrb182y13h-python3.7-moto-1.3.8.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/w7vfwa40gj71apdvniavxs0gah1927ia-python3.7-buildbot-2.3.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/w7vfwa40gj71apdvniavxs0gah1927ia-python3.7-buildbot-2.3.1.drv' failed

@FRidh will probably have better opinions on how to go about this.

@FRidh
Copy link
Member

FRidh commented Jul 19, 2019

would be to pin pythonPackages.jsonschema_2 and go to each package that still requires it and amend them.

We cannot have multiple versions of the same package in python-packages.nix because that leads to collisions when using them together.

@FRidh
Copy link
Member

FRidh commented Jul 19, 2019

Here the typical approach is to be a bit patient, and when a significant majority is compatible just make the switch and break the rest.

@jonringer
Copy link
Contributor

jonringer commented Jul 19, 2019

oh right, because they would both exist on PYTHONPATH, and wouldn't be deterministic in which one would be loaded first....

@FRidh FRidh self-assigned this Jul 20, 2019
@multun
Copy link
Member

multun commented Jul 31, 2019

@FRidh

We cannot have multiple versions of the same package in python-packages.nix because that leads to collisions when using them together.

Isn't it somehow already the case ?

(I'm not quite sure about what I say, I'm pretty much a nix newbie)

@FRidh
Copy link
Member

FRidh commented Jul 31, 2019

I know. That went in but should not have gone in.

@jonringer jonringer mentioned this pull request Aug 3, 2019
5 of 10 tasks complete
@worldofpeace
Copy link
Member

worldofpeace commented Oct 1, 2019

@FRidh ping. Most pertinent thing here is @costrouc updated jsonschema and its gone unnoticed for months...

@FRidh
Copy link
Member

FRidh commented Oct 1, 2019

It's not gone unnoticed for months. It's just that we simply cannot update jsonschema because it would break too many other reverse dependencies. Now we could investigate again.

@FRidh FRidh mentioned this pull request Dec 19, 2019
0 of 10 tasks complete
@jonringer
Copy link
Contributor

jonringer commented Jan 9, 2020

most of these commits have gone very stale.

bumping @costrouc

@worldofpeace
Copy link
Member

worldofpeace commented Jan 9, 2020

I think they might have got integrated somehow?
I hope this hasn't discouraged @costrouc in any way from contributing. I simply can't remember what exactly happened, perhaps it was an issue with me not being informed enough about python-unstable?

Anyways, I will close it because it's out of date and conflicting.
I hope any similar situation shouldn't occur anymore.

@jonringer
Copy link
Contributor

jonringer commented Jan 9, 2020

it was paused for the longest time because the jsonschema major bump broke a lot of packages that didn't support >=3.0.

Then in october, python 3.8 was released and much of the python landscape changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.