Prefer wheels over source distribution

[dimbleby]

#5385 against recent code, and with a couple of minor improvements (but full credit @gpfv)

In particular I have

• accounted for the possibility that a single wheel can declare support for multiple pyversion / abi / platform (the .split(".") stuff)
• rearranged the test script so that it tests both cases independent of the platform on which the tests happen to be executing

Per #5385 this gives a meaningful performance improvement in real world cases eg on my laptop the locking of dependencies per https://github.com/lincolnloop/python-package-manager-shootout comes down from about 120s to about 60s.

It will also sidestep at least some of the many "I am trying to install a package whose pep517 build from source is broken" issues.

[dimbleby]

It probably ought to be safe in this code to take the first wheel always, come to think of it, whether compatible with the current platform or not. poetry is extracting metadata that ought to be the same regardless of which wheel it reads it from. Indeed it relies on that being true, else the solution that it builds is not a cross-platform solution after all.

That takes locking the shootout down from ~60s to ~45s for me without obviously breaking anything.

Views on whether that's a good idea?

[neersighted]

I don't see any issue extracting metadata from the first wheel -- that is how PyPI works, and if metadata is different on one platform to another that is on the packagers.

[dimbleby]

I don't see any issue extracting metadata from the first wheel -- that is how PyPI works, and if metadata is different on one platform to another that is on the packagers.

I mostly agree.

The really principled change to this code would just be always to look at literally any of the wheels, but doing that broke some interesting unit test. I've expanded a comment explaining why we still look for py2 and py3 universal wheels.

It is plausible that this could all be removed: recent isort has dropped python2 altogether, even 4.3.21 published a single py2.py3 wheel, if you're feeling lucky about the state of the python world then this whole code branch could just look at wheels[0] and be done.

[neersighted]

Ultimately we are going to have to make a similar decision about how we gather metadata (or evolve the solver) for PyPI as well, as the metadata that is available is intended to be per-distfile in the long term per PEP 658.

I don't see the harm in preferring pure Python and then platform wheels for now (as opposed to the current 'universal wheel, py-version wheel, platform wheel') as ultimately it's a very minor change. Simply taking wheels[0] might be something we do in the long run, but seems too high-risk at this current time.

[neersighted]

It increases the scope of this change a bit, but while you're in here would you be willing to simplify this logic to prefer a Python-only wheel (e.g. universal or py3), platform_specific_wheels[0], and then sdist?

[dimbleby]

I don't understand what you're asking for that's different from what the code already does

[neersighted]

Right now the logic prioritizes a universal wheel (py2.py3), then a py2 wheel (dead code), then a py3 wheel. It could be significantly simplified to instead look for any Python 3 compatible pure Python wheel ((py2\.)?py3, though that regex is a simplficiation) wheel as the first step, removing a large swathe of wheel-selection code.

[dimbleby]

the bit of the code that's complicated is the bit that deals with "there's both a py2 and py3 wheel" eg per isort 4.3.4. Dropping that breaks the testcase that verifies that it works.

In what sense is py2 wheels dead code? Did poetry drop support for managing projects that want to be python2 compatible?

I suspect we're probably at cross purposes here, but I still don't understand what you're after. Feel free to push to this MR, but perhaps it's better handled separately.

[neersighted]

Poetry has indeed dropped support for managing projects using Python 2.7 (https://python-poetry.org/blog/announcing-poetry-1.2.0/#dropping-support-for-managing-python-27-projects) -- though not all relevant code was removed in time for the 1.2 release.

We were forced to make this change due to our dependencies (some of which need to run inside the target environment) dropping Python 2.7 support, but given it's EOL as of nearly 24 months ago (after a years-long deprecation), I think it was overdue in Poetry anyway.

[dimbleby]

There's probably a fair bit more to do than just tidy up this code if that's to be taken seriously.

There are still quite a few places in poetry-core that default to ~2.7 || >=3.4 for instance, and many of the test scripts in both projects have a similar python version constraint.

I think I'd prefer to let this MR stand as it is, I don't feel in the mood to chase down the inevitable unit test failures if I remove the py2 code.

[neersighted]

Fair enough -- we'll leave that as an exercise for the next person (and indeed, the last PR to attempt to excise Python 2 support stalled due to the extensive entanglement in the test suite).

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.