Python 3.12: ModuleNotFoundError: No module named 'setuptools'

[hugovk]

The second and final Python 3.12 release candidate is out! 🚀

Call to action

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.12 compatibilities during this phase, and where necessary publish Python 3.12 wheels on PyPI to be ready for the final release of 3.12.0.

---

However, testing on Python 3.12.0rc2:

❯ tox -e py312
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_wheel> python -I -m pip install wheel
.pkg: prepare_metadata_for_build_wheel> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py312: install_package> python -I -m pip install --force-reinstall --no-deps /private/tmp/dill/.tox/.tmp/package/1/dill-0.3.2.dev0.tar.gz
py312: commands[0]> .tox/py312/bin/python setup.py build
Traceback (most recent call last):
  File "/private/tmp/dill/setup.py", line 19, in <module>
    from setuptools import setup
ModuleNotFoundError: No module named 'setuptools'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/dill/setup.py", line 22, in <module>
    from distutils.core import setup
ModuleNotFoundError: No module named 'distutils'
py312: exit 1 (0.02 seconds) /private/tmp/dill> .tox/py312/bin/python setup.py build pid=92651
.pkg: _exit> python /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  py312: FAIL code 1 (3.53=setup[3.52]+cmd[0.02] seconds)
  evaluation failed :( (3.57 seconds)

This will be due to the setuptools changes in Python 3.12:

easy_install, pkg_resources, setuptools and distutils are no longer provided by default in environments created with venv or bootstrapped with ensurepip, since they are part of the setuptools package. For projects relying on these at runtime, the setuptools project should be declared as a dependency and installed separately (typically, using pip).

https://docs.python.org/dev/whatsnew/3.12.html#ensurepip

See also gitpython-developers/GitPython#1640 (comment) for more advice.

[EliahKagan]

I am unable to reproduce this at the tip of the master branch (currently b66411e). Does this happen at the tip of master, or with any recent releases? This part of your output suggests you may be testing a very old version:

py312: install_package> python -I -m pip install --force-reinstall --no-deps /private/tmp/dill/.tox/.tmp/package/1/dill-0.3.2.dev0.tar.gz

dill 0.3.2 was released in June 2020, with 0.3.3 in November 2020. So I don't think tox runs on any recent dill should show that. The corresponding part of my tox -e py312 output shows:

py312: install_package> python -I -m pip install --force-reinstall --no-deps /home/ek/repos-wsl/dill/.tox/.tmp/package/2/dill-0.3.8.dev0.tar.gz

And for my next line, instead of:

py312: commands[0]> .tox/py312/bin/python setup.py build

I have:

py312: commands[0]> .tox/py312/bin/python -m pip install .

This is what I expect, because tox.ini does not directly use setup.py since #469 (per this diff).

---

With that said, even if I am understanding correctly and the problem you've reported here was effectively fixed in #469, it seems to me that there is at least one thing that should be updated, relating to Python 3.12 not having setuptools installed by default:

dill/dill/tests/__init__.py

Lines 10 to 11 in b66411e

	$ python setup.py build
	$ python setup.py install

However, that cannot be the cause of test failures, because it appears in a docstring (that is not a doctest).

[mmckerns]

@EliahKagan: Thanks for your posting. I was wondering the same thing. Good catch on the outdated docs.

[EliahKagan]

No problem--and thanks for the fix for that (#621)!

I've noticed that the new instructions give an installation command that assumes the current directory is the location of that file (they refer to the repository root as ../..). This differs from the old instructions, which assumed they were run in the root of the repository (where seup.py is).

Although I find this odd, I'm reluctant to propose that it be changed, in case it is intentional.

[hugovk]

Ah sorry, I failed to update my branch properly from upstream and it was very out of date. All is good when updating it.

At least it helped update the docs :) Sorry again for the noise, and thank you for testing 3.12 pre-releases!

[mmckerns]

I've noticed that the new instructions give an installation command that assumes the current directory is the location of that file (they refer to the repository root as ../..). This differs from the old instructions, which assumed they were run in the root of the repository (where seup.py is).

Although I find this odd, I'm reluctant to propose that it be changed, in case it is intentional.

Yeah, it's intentional. I thought it a bit odd too, but I think it more correct. Maybe it'd be better to say one should "cd to the package root" then python -m pip install . -- that might be better...