Remove bundled setuptools #95299

sbidoul · 2022-07-26T20:37:59Z

Feature or enhancement

Remove the bundled setuptools so that ensurepip and python -m venv only installs pip.

Context

The setup.py install command of setuptools is deprecated.

However, in an environment where setuptools is installed but wheel is not (such as one created with python -m venv), pip falls back on running the deprecated and non-standard setup.py install.

Since version 22.1 pip works correctly by default in environments where setuptools is not installed, by enabling its PEP 517 mode automatically, leading to unsurprising installations in most cases.

So, in order to progressively expose more users to standard-based installation workflows, we (the pip team) would like that virtual environments are created without setuptools by default.

Users faced with failing installations following this change (likely due to packages with customized setup.py that do not support building a wheel) can easily pip install setuptools to solve the issue.

Previous discussion

pypa/pip#8102 (comment) and following comments has some more context.

Linked PRs

The text was updated successfully, but these errors were encountered:

sbidoul · 2022-07-26T20:40:06Z

@pablogsal do you think it is still acceptable to do this in 3.11? If positive we can likely land a PR on time for RC1.

pablogsal · 2022-07-26T20:41:56Z

Unfortunately I don't feel comfortable landing this so late in the release cycle, specially so close to RC1 and given the current stability of 3.11 :(

This will need to wait until 3.12.

rafrafek · 2022-11-07T15:09:39Z

It feels strange that setuptools is included when using -m venv, but wheel is not included.

pradyunsg · 2023-01-14T22:44:15Z

I've filed #101039 for this (almost had a number with three 0s), targetting 3.12.

edmorley · 2023-02-23T12:49:23Z

Since version 22.1 pip works correctly by default in environments where setuptools is not installed, by enabling its PEP 517 mode automatically, leading to unsurprising installations in most cases.

Do you have some examples of cases where the PEP 517 mode doesn't work for legacy packages? :-)

FFY00 · 2023-03-04T08:05:56Z

PEP 517 is backwards compatible, it should work on legacy packages, the opposite is the issue, pip trying to use the legacy path for packages that need PEP 517.

…directly" This reverts commit 41c5a66.

# This is the 1st commit message: pythongh-95299: Rework test_cppext.py to not invoke setup.py directly # This is the commit message #2: Add tests/cppextdata data to `TESTSUBDIRS` # This is the commit message #3: Revert "Add tests/cppextdata data to `TESTSUBDIRS`" This reverts commit 635492e.

) * gh-95299: Rework test_cppext.py to not invoke setup.py directly * Add tests/cppextdata data to `TESTSUBDIRS` * Revert "Add tests/cppextdata data to `TESTSUBDIRS`" This reverts commit 635492e. * Revert "gh-95299: Rework test_cppext.py to not invoke setup.py directly" This reverts commit 41c5a66. * Build and install the extension in a temporary directory instead * Pull in wheels for setuptools and wheel for testing extension builds

* main: pythongh-103479: [Enum] require __new__ to be considered a data type (pythonGH-103495) pythongh-103365: [Enum] STRICT boundary corrections (pythonGH-103494) pythonGH-103488: Use return-offset, not yield-offset. (pythonGH-103502) pythongh-103088: Fix test_venv error message to avoid bytes/str warning (pythonGH-103500) pythonGH-103082: Turn on branch events for FOR_ITER instructions. (python#103507) pythongh-102978: Fix mock.patch function signatures for class and staticmethod decorators (python#103228) pythongh-103462: Ensure SelectorSocketTransport.writelines registers a writer when data is still pending (python#103463) pythongh-95299: Rework test_cppext.py to not invoke setup.py directly (python#103316)

…101039) Remove the bundled setuptools wheel from ensurepip, and stop installing setuptools in environments created by venv. Co-Authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM> Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>

pradyunsg · 2023-04-18T04:45:38Z

https://docs.python.org/3.12/library/venv.html#an-example-of-extending-envbuilder is the final piece of work here -- that example needs to be updated to use something more modern or otherwise behave differently to showcase extending behaviours of the class.

…python#103316) * pythongh-95299: Rework test_cppext.py to not invoke setup.py directly * Add tests/cppextdata data to `TESTSUBDIRS` * Revert "Add tests/cppextdata data to `TESTSUBDIRS`" This reverts commit 635492e. * Revert "pythongh-95299: Rework test_cppext.py to not invoke setup.py directly" This reverts commit 41c5a66. * Build and install the extension in a temporary directory instead * Pull in wheels for setuptools and wheel for testing extension builds

…ripts

…H-103613)

* main: (24 commits) pythongh-98040: Move the Single-Phase Init Tests Out of test_imp (pythongh-102561) pythongh-83861: Fix datetime.astimezone() method (pythonGH-101545) pythongh-102856: Clean some of the PEP 701 tokenizer implementation (python#103634) pythongh-102856: Skip test_mismatched_parens in WASI builds (python#103633) pythongh-102856: Initial implementation of PEP 701 (python#102855) pythongh-103583: Add ref. dependency between multibytecodec modules (python#103589) pythongh-83004: Harden msvcrt further (python#103420) pythonGH-88342: clarify that `asyncio.as_completed` accepts generators yielding tasks (python#103626) pythongh-102778: IDLE - make sys.last_exc available in Shell after traceback (python#103314) pythongh-103582: Remove last references to `argparse.REMAINDER` from docs (python#103586) pythongh-103583: Always pass multibyte codec structs as const (python#103588) pythongh-103617: Fix compiler warning in _iomodule.c (python#103618) pythongh-103596: [Enum] do not shadow mixed-in methods/attributes (pythonGH-103600) pythonGH-100530: Change the error message for non-class class patterns (pythonGH-103576) pythongh-95299: Remove lingering setuptools reference in installer scripts (pythonGH-103613) [Doc] Fix a typo in optparse.rst (python#103504) pythongh-101100: Fix broken reference `__format__` in `string.rst` (python#103531) pythongh-95299: Stop installing setuptools as a part of ensurepip and venv (python#101039) pythonGH-103484: Docs: add linkcheck allowed redirects entries for most cases (python#103569) pythongh-67230: update whatsnew note for csv changes (python#103598) ...

And bump some github action dependencies. `setuptools` is added explicitly as a dependency to support python 3.12. > Python 3.12 release notes https://docs.python.org/3/whatsnew/3.12.html > > python/cpython#95299: Do not pre-install setuptools in virtual environments > created with venv. This means that distutils, setuptools, pkg_resources, and > easy_install will no longer available by default; to access these run pip > install setuptools in the activated virtual environment.

And bump some github action dependencies. `invoke` is bumped to `2.2.0` to support python 3.12. `setuptools` is added explicitly as a dependency to support python 3.12. > Python 3.12 release notes https://docs.python.org/3/whatsnew/3.12.html > > python/cpython#95299: Do not pre-install setuptools in virtual environments > created with venv. This means that distutils, setuptools, pkg_resources, and > easy_install will no longer available by default; to access these run pip > install setuptools in the activated virtual environment.

python/cpython#95299

This change expands our python integration tests to load and exercise libcrypto-linking 3rd party modules. Cases covered include statically linked AWS-LC (default AWS CRT) and dynamically linked OpenSSL (AWS CRT, PyOpenSSL, PyCA). We also test these integrations with python's AWS-LC built in FIPS mode. Due to the pre-release nature of python 3.13 and 3.14, there are a few caveats for later versions: 3.12 introduced [a change](python/cpython#95299) to drop `setuptools` from default virtual environments, causing installation issues for PyCA and PyOpenSSL on 3.13+. To work around this, we allow installation failure for those dependencies on newer (currently both 3.13 and 3.14 are pre-release) and exit early from the relevant test cases. We also encountered some issues installing CRT bindings from source on 3.12+, so for those versions we install the pre-compiled wheel module from PyPI, which uses AWS-LC under the hood.

Since python 3.12, pkg_resources has been moved to setuptools. Also, it is deprecated. - Kozea/Radicale#1184 - mu-editor/mu#2485 - python/cpython#95299 Signed-off-by: Shohei Maruyama <cheat.sc.linux@outlook.com>

Since python 3.12, pkg_resources has been moved to setuptools. Also, it is deprecated. - Kozea/Radicale#1184 - mu-editor/mu#2485 - python/cpython#95299 $ python -m ttfautohint --help Traceback (most recent call last): File "/usr/lib/python3.12/site-packages/ttfautohint/_version.py", line 2, in <module> from pkg_resources import get_distribution, DistributionNotFound ModuleNotFoundError: No module named 'pkg_resources' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<frozen runpy>", line 189, in _run_module_as_main File "<frozen runpy>", line 148, in _get_module_details File "<frozen runpy>", line 112, in _get_module_details File "/usr/lib/python3.12/site-packages/ttfautohint/__init__.py", line 12, in <module> from ttfautohint._version import __version__ File "/usr/lib/python3.12/site-packages/ttfautohint/_version.py", line 4, in <module> except (ImportError, DistributionNotFound): ^^^^^^^^^^^^^^^^^^^^ NameError: name 'DistributionNotFound' is not defined Signed-off-by: Shohei Maruyama <cheat.sc.linux@outlook.com>

This PR fixes GitHub actions for unit tests by adding setuptools as they are not pre-installed anymore. python/cpython#95299

* Update swift from branch 'master' to 9c38d756e0233eaa5f9e26066756eea9ce6a11ad - Merge "Use entry_points for server executables" - Use entry_points for server executables The old [files]scripts method of specifying executable Python scripts triggers some legacy easy-install-like mode for editable installs, which relies on pkg_resources. Recent versions of setuptools (67.5.0+) have started emitting warnings when importing pkg_resources, which in turn cause quite noticeable slowdowns in process startup. This is particularly prominant on py312, which stopped pre-installing (an often older version of) setuptools in new venvs. See also: - python/cpython#95299 - pypa/setuptools#3843 - pypa/setuptools#3966 Now, use [entry_points]console_scripts to specify these executables, which does not use pkg_resources in the generated script files. Change-Id: Ifcc8138e7b55d5b82bea0d411ec6bfcca2c77c83

The old [files]scripts method of specifying executable Python scripts triggers some legacy easy-install-like mode for editable installs, which relies on pkg_resources. Recent versions of setuptools (67.5.0+) have started emitting warnings when importing pkg_resources, which in turn cause quite noticeable slowdowns in process startup. This is particularly prominant on py312, which stopped pre-installing (an often older version of) setuptools in new venvs. See also: - python/cpython#95299 - pypa/setuptools#3843 - pypa/setuptools#3966 Now, use [entry_points]console_scripts to specify these executables, which does not use pkg_resources in the generated script files. Change-Id: Ifcc8138e7b55d5b82bea0d411ec6bfcca2c77c83

sbidoul added the type-feature A feature request or enhancement label Jul 26, 2022

sbidoul mentioned this issue Jul 26, 2022

Raise a warning when pip falls back to the "legacy" direct install code pypa/pip#8102

Closed

hugovk added the 3.12 bugs and security fixes label Jul 26, 2022

cjolowicz mentioned this issue Aug 28, 2022

Consider preparing venv and virtualenv sessions without setuptools and wheel wntrblm/nox#557

Open

pradyunsg mentioned this issue Sep 23, 2022

[POC] gh-85454: Extract and inject distutils from setuptools whl #95254

Closed

bedevere-bot mentioned this issue Jan 14, 2023

gh-95299: Stop installing setuptools as a part of ensurepip and venv #101039

Merged

pradyunsg added the topic-ensurepip label Feb 21, 2023

hugovk mentioned this issue Feb 24, 2023

gh-102202: Update bundled setuptools to 67.4.0 #102208

Closed

pradyunsg added a commit to pradyunsg/cpython that referenced this issue Apr 6, 2023

pythongh-95299: Rework test_cppext.py to not invoke setup.py directly

41c5a66

bedevere-bot mentioned this issue Apr 6, 2023

gh-95299: Rework test_cppext.py to not invoke setup.py directly #103316

Merged

pradyunsg added a commit to pradyunsg/cpython that referenced this issue Apr 6, 2023

Revert "pythongh-95299: Rework test_cppext.py to not invoke setup.py …

d79d6a6

…directly" This reverts commit 41c5a66.

pradyunsg added a commit to pradyunsg/cpython that referenced this issue Apr 9, 2023

pythongh-95299: Rework test_cppext.py to use explicitly in a tempdir

02d8a63

pradyunsg added a commit to pradyunsg/cpython that referenced this issue Apr 9, 2023

pythongh-95299: Rework test_cppext.py to use explicitly in a tempdir

846a148

bedevere-bot mentioned this issue Apr 18, 2023

gh-95299: Remove lingering setuptools reference in installer scripts #103613

Merged

zooba added a commit to zooba/cpython that referenced this issue Apr 18, 2023

pythongh-95299: Remove lingering setuptools reference in installer sc…

c909a07

…ripts

zooba added a commit that referenced this issue Apr 18, 2023

gh-95299: Remove lingering setuptools reference in installer scripts (G…

78cac52

…H-103613)

pfmoore mentioned this issue Apr 22, 2023

gh-103559: Update bundled pip version to 23.1.1 #103560

Merged

ivantodorovich mentioned this issue Apr 25, 2024

✨ Support python 3.12 and bump some dependencies camptocamp/odoo-project-tools#42

Merged

ataylor32 mentioned this issue Apr 28, 2024

Don't depend on pkg_resources jazzband/django-user-sessions#185

Open

kstreser mentioned this issue May 4, 2024

[BUG] Missing pkg_resources for Python >= 3.12 bergercookie/syncall#113

Open

Sithuk mentioned this issue May 6, 2024

Upgrade from Ubuntu 22.04 to 24.04 broke pyznap yboetz/pyznap#106

Open

jmahlik mentioned this issue May 9, 2024

[SETUP-BUG] Missing setuptools dependency mlflow/mlflow#11330

Open

chedabob mentioned this issue May 16, 2024

Fix documentation builds Apadmi-Engineering/Mockzilla#71

Merged

galactics added a commit to galactics/beyond that referenced this issue May 19, 2024

build : setuptools necessary for installation inside a virtualenv

430c900

python/cpython#95299

WillChilds-Klein mentioned this issue May 20, 2024

Add integration tests for OpenSSL-linking 3p modules aws/aws-lc#1587

Merged

bersbersbers mentioned this issue Jun 2, 2024

python 3.12 removes pkg_resources klen/pylama#249

Open

Th3Bobo mentioned this issue Jun 5, 2024

Python 3.12 support? GregoryAM-SP/The-Minecraft-Overviewer#74

Closed

devon-mar mentioned this issue Jun 9, 2024

setuptools is not listed as a dependency nornir-automation/nornir#917

Closed

lesteve mentioned this issue Jun 14, 2024

MAINT Remove setuptools dependency from tests scikit-learn/scikit-learn#29261

Merged

cheat-sc mentioned this issue Jun 15, 2024

python: ttfautohint: Python 3.12 support fonttools/ttfautohint-py#22

Open

elikoga mentioned this issue Jun 17, 2024

Install setuptools into venv if python >= 3.12 for backwards compatibility flyingcircusio/appenv#54

Open

huwjenkins mentioned this issue Jun 20, 2024

Suggestion: Homebrew formula for gtk3 (now GTK4) installation on macOS (or Linux) pemsley/coot#33

Open

leplatrem mentioned this issue Jul 2, 2024

Add Python 3.12 support Kinto/kinto-http.py#372

Merged

cclauss mentioned this issue Jul 3, 2024

RFE: Replace use of pkg_resources with importlib.metadata python-babel/babel#861

Open

jirimoravcik mentioned this issue Jul 4, 2024

chore: add setuptools to opt deps apify/apify-shared-python#22

Merged

jirimoravcik added a commit to apify/apify-shared-python that referenced this issue Jul 4, 2024

chore: add setuptools to opt deps (#22)

a9a0bdc

This PR fixes GitHub actions for unit tests by adding setuptools as they are not pre-installed anymore. python/cpython#95299

Morikko mentioned this issue Jul 8, 2024

No module named 'pkg_resources' when using pipx owdevel/ytdl-nfo#30

Closed

jonathan343 mentioned this issue Jul 19, 2024

Fix CI for Python >= 3.12 boto/botocore#3223

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove bundled setuptools #95299

Remove bundled setuptools #95299

sbidoul commented Jul 26, 2022 •

edited by bedevere-bot

Loading

sbidoul commented Jul 26, 2022

pablogsal commented Jul 26, 2022

rafrafek commented Nov 7, 2022

pradyunsg commented Jan 14, 2023

edmorley commented Feb 23, 2023

FFY00 commented Mar 4, 2023

pradyunsg commented Apr 18, 2023

Remove bundled setuptools #95299

Remove bundled setuptools #95299

Comments

sbidoul commented Jul 26, 2022 • edited by bedevere-bot Loading

Feature or enhancement

Context

Previous discussion

Linked PRs

sbidoul commented Jul 26, 2022

pablogsal commented Jul 26, 2022

rafrafek commented Nov 7, 2022

pradyunsg commented Jan 14, 2023

edmorley commented Feb 23, 2023

FFY00 commented Mar 4, 2023

pradyunsg commented Apr 18, 2023

sbidoul commented Jul 26, 2022 •

edited by bedevere-bot

Loading