ensurepip: add configure --with-wheel-pkg-dir=PATH to get wheel packages from a system directory #87022
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = None closed_at = <Date 2021-01-21.09:50:21.117> created_at = <Date 2021-01-07.11:21:06.475> labels = ['library', '3.10'] title = 'ensurepip: add configure --with-wheel-pkg-dir=PATH to get wheel packages from a system directory' updated_at = <Date 2021-01-21.09:50:21.117> user = 'https://github.com/vstinner'
activity = <Date 2021-01-21.09:50:21.117> actor = 'vstinner' assignee = 'none' closed = True closed_date = <Date 2021-01-21.09:50:21.117> closer = 'vstinner' components = ['Library (Lib)'] creation = <Date 2021-01-07.11:21:06.475> creator = 'vstinner' dependencies =  files =  hgrepos =  issue_num = 42856 keywords = ['patch'] message_count = 7.0 messages = ['384577', '384578', '384579', '385027', '385031', '385032', '385353'] nosy_count = 7.0 nosy_names = ['ncoghlan', 'vstinner', 'christian.heimes', 'mcepl', 'dstufft', 'pradyunsg', 'hroncok'] pr_nums = ['24151', '24210'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue42856' versions = ['Python 3.10']
The text was updated successfully, but these errors were encountered:
The Fedora packaging policy recommends to "unbundle" bundled dependencies.
"Fedora packages SHOULD make every effort to avoid having multiple, separate, upstream projects bundled together in a single package."
The main motivation is to ease updates when fix serious vulnerabilities (less packaging work).
The ensurepip package contains bundled wheel packages of setuptools and pip:
$ ls Lib/ensurepip/_bundled/ pip-20.2.3-py2.py3-none-any.whl setuptools-47.1.0-py3-none-any.whl
The Fedora python3 package doesn't contain the ensurepip._bundled package:
$ python3 Python 3.9.1 (default, Dec 8 2020, 00:00:00) >>> import ensurepip._bundled ModuleNotFoundError: No module named 'ensurepip._bundled'
Instead, a separated RPM package python-pip-wheel provides wheel packages in /usr/share/python-wheels/ directory:
$ ls /usr/share/python-wheels/ pip-20.2.2-py2.py3-none-any.whl* setuptools-49.1.3-py3-none-any.whl* wheel-0.34.2-py2.py3-none-any.whl*
Fedora has a downstream patch on ensurepip (written by Miro Hrončok) to always use /usr/share/python-wheels/:
Fedora has packages of 9 CPython versions: 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10.
Having a separated package for wheel packages allows us to upgrade a single package (python-pip-wheel) for setuptools/pip bugfix or security vulnerability.
I propose to add a new --with-wheel-pkg-dir=PATH option to the ./configure script. If used, ensurepip will only use wheel packages from this directory. Otherwise, the existing code is unchanged. In short, the behavior is unchanged, unless the option is used explicitly.
If a directory is specified but wheel packages are missing, ensurepip fails.
If the directory contains multiple wheel packages of different versions, the most recent version is used of each package.
Note: In practice, the Fedora package only provides a single wheel package of each Python module. But I propose to make the Python upstream code as generic as possible.
I'm working on a pull request to implement this.
Downstream Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=1874803
Fedora (Miro) already contributed to ensurepip to make ensurepip less dependent on pip internals:
This change was already related to Fedora downstream change to get wheel packages from a different directory. Fedora can use a different pip version (older or more recent) than ensurepip._bundled.
An alternative is to find packages in all directories and pick the most recent version.
Example with a specified directory *and* ensurepip._bundled is available:
Most recent versions:
Problem: I'm not sure that pip is fully compatible with any setuptools version.
We (SUSE) have updated versions of the wheels as special Sources, and then this in the %prep stage of our SPEC file:
A bit of manual work required, but it doesn't lead to so incredible convoluted constructs as I see in Fedora (nothing against it, but our build system is already convoluted enough).