[MNT]: install on Python 3.12.0b3

[bersbersbers]

Summary

pip install matplotlib fails on Python 3.12.0b3, see pypa/setuptools#3935. This used to be blocked on numpy/numpy#23808, but that is now fixed: pip install https://anaconda.org/scientific-python-nightly-wheels/numpy/2.0.0.dev0/download/numpy-2.0.0.dev0-cp312-cp312-win_amd64.whl works on Windows to install numpy 2.0.0.dev0. However, even then installation of matplotlib still fails.

Is there a way to install matplotlib on Python 3.12 betas?

Proposed fix

No response

[oscargus]

It seems to vary if it is possible or not. The tests did pass on Python 3.12.0a7 #24711 and some work is done to sort out 3.12 issues: #26154

What error message are you getting now?

[bersbersbers]

I get distutils errors when it tries installing numpy==1.25.0 (although a numpy dev version is installed already.)

Microsoft Windows [Version 10.0.19044.2965]
(c) Microsoft Corporation. All rights reserved.

C:\Users\bers>python -m pip list
Package Version
------- ----------
numpy   2.0.0.dev0
pip     23.1.2

C:\Users\bers>python --version
Python 3.12.0b3

C:\Users\bers>python -m pip list
Package Version
------- ----------
numpy   2.0.0.dev0
pip     23.1.2

C:\Users\bers>python -m pip install matplotlib
Collecting matplotlib
  Using cached matplotlib-3.7.1.tar.gz (38.0 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 2
  ╰─> [113 lines of output]
      Collecting certifi>=2020.06.20
        Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)
      Collecting oldest-supported-numpy
        Using cached oldest_supported_numpy-2022.11.19-py3-none-any.whl (4.9 kB)
      Collecting pybind11>=2.6
        Using cached pybind11-2.10.4-py3-none-any.whl (222 kB)
      Collecting setuptools_scm>=7
        Using cached setuptools_scm-7.1.0-py3-none-any.whl (43 kB)
      Collecting numpy (from oldest-supported-numpy)
        Using cached numpy-1.25.0.tar.gz (10.4 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
      ERROR: Exception:
      Traceback (most recent call last):
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\cli\base_command.py", line 169, in exc_logging_wrapper
          status = run_func(*args)
                   ^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\cli\req_command.py", line 248, in wrapper
          return func(self, options, args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\commands\install.py", line 377, in run
          requirement_set = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 92, in resolve
          result = self._result = resolver.resolve(
                                  ^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 546, in resolve
          state = resolution.resolve(requirements, max_rounds=max_rounds)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 427, in resolve
          failure_causes = self._attempt_to_pin_criterion(name)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 239, in _attempt_to_pin_criterion
          criteria = self._get_updated_criteria(candidate)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 230, in _get_updated_criteria
          self._add_to_criteria(criteria, requirement, parent=candidate)
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
          if not criterion.candidates:
                 ^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\resolvelib\structs.py", line 156, in __bool__
          return bool(self._sequence)
                 ^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__
          return any(self)
                 ^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in <genexpr>
          return (c for c in iterator if id(c) not in self._incompatible_ids)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built
          candidate = func()
                      ^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 206, in _make_candidate_from_link
          self._link_candidate_cache[link] = LinkCandidate(
                                             ^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 293, in __init__
          super().__init__(
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
          self.dist = self._prepare()
                      ^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 225, in _prepare
          dist = self._prepare_distribution()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 304, in _prepare_distribution
          return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\operations\prepare.py", line 516, in prepare_linked_requirement
          return self._prepare_linked_requirement(req, parallel_builds)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\operations\prepare.py", line 631, in _prepare_linked_requirement
          dist = _get_prepared_distribution(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\operations\prepare.py", line 69, in _get_prepared_distribution
          abstract_dist.prepare_distribution_metadata(
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\distributions\sdist.py", line 48, in prepare_distribution_metadata
          self._install_build_reqs(finder)
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\distributions\sdist.py", line 118, in _install_build_reqs
          build_reqs = self._get_build_requires_wheel()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\distributions\sdist.py", line 95, in _get_build_requires_wheel
          return backend.get_requires_for_build_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_internal\utils\misc.py", line 692, in get_requires_for_build_wheel
          return super().get_requires_for_build_wheel(config_settings=cs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 166, in get_requires_for_build_wheel
          return self._call_hook('get_requires_for_build_wheel', {
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 321, in _call_hook
          raise BackendUnavailable(data.get('traceback', ''))
      pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bers\.pyenv\pyenv-win\versions\3.12.0b3\Lib\importlib\__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1216, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1237, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 841, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 994, in exec_module
        File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
        File "C:\Users\bers\AppData\Local\Temp\pip-build-env-hqcdn_f2\overlay\Lib\site-packages\setuptools\__init__.py", line 10, in <module>
          import distutils.core
      ModuleNotFoundError: No module named 'distutils'

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

C:\Users\bers>

[tacaswell]

Try using

pip install -v matplotlib --no-build-isolation

you may have to manually install some additional dependencies, but you won't have to deal with a background environment.

[bersbersbers]

This makes it go much further:

python -m pip install setuptools pybind11 wheel
python -m pip install -v matplotlib --no-build-isolation

Still, it gives (among others)

...
Discarding https://files.pythonhosted.org/packages/b7/65/d6e00376dbdb6c227d79a2d6ec32f66cfb163f0cd924090e3133a4f85a11/matplotlib-3.7.1.tar.gz (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.8): Requested matplotlib from https://files.pythonhosted.org/packages/b7/65/d6e00376dbdb6c227d79a2d6ec32f66cfb163f0cd924090e3133a4f85a11/matplotlib-3.7.1.tar.gz has inconsistent version: expected '3.7.1', but metadata has '0.0.0'
...
Discarding https://files.pythonhosted.org/packages/65/c2/34158ff731a12802228434e8d17d2ebb5097394ab9d065205cc262cf2a6f/matplotlib-3.7.0.tar.gz (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.8): Requested matplotlib from https://files.pythonhosted.org/packages/65/c2/34158ff731a12802228434e8d17d2ebb5097394ab9d065205cc262cf2a6f/matplotlib-3.7.0.tar.gz has inconsistent version: expected '3.7.0', but metadata has '0.0.0'
...
ModuleNotFoundError: No module named 'mesonpy'

python -m pip install mesonpy does not work, either:

C:\Users\bers>python -m pip install mesonpy
Collecting mesonpy
  Using cached MesonPy-0.0.1.tar.gz (14 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\bers\AppData\Local\Temp\pip-install-j3rgaf3x\mesonpy_6bb5e49fe83f4323a455e3fcdf4a0764\setup.py", line 4, in <module>
          with open('requirements.txt') as f:
               ^^^^^^^^^^^^^^^^^^^^^^^^
      FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[tacaswell]

We do not directly depend or use mesonpy so I am not sure why you need it...maybe kiwisolver uses it?

[ianthomas23]

ContourPy uses meson-python but not mesonpy, which is a totally unrelated package that was released in 2017 and then abandoned. Please try

python -m pip install meson-python

instead.

Using python -m pip install -v matplotlib --no-build-isolation might be good advice from a Matplotlib/NumPy point of view, but if this requires dependencies for which there are no python 3.12 wheels, such as ContourPy, you will be building them from source also without build isolation so you will have to preinstall the build dependencies of those packages without even knowing what they are or what their build dependencies are. That might be a long, painful, iterative process.

What I would try is:

python -m pip install --upgrade pip
python -m pip install contourpy

before

python -m pip install setuptools pybind11 wheel
python -m pip install -v matplotlib --no-build-isolation

[bersbersbers]

Okay, I guess I will have to wait until a bunch of other projects fix their setup routines for Python 3.12 or make wheels available. For pip install contourpy, I am getting Could not parse vswhere.exe output - I have VS 2022 (17.6.4) installed, but I do not go down that rabbit hole to compile dependencies of dependencies :)

[ianthomas23]

Very sensible!

For completeness, I have just added Github Action runs for contourpy on Python 3.12 which build numpy from sdist before building and testing contourpy in the usual manner. Links to relevant CI runs:

• Linux https://github.com/contourpy/contourpy/actions/runs/5338853882/jobs/9676797971
• macOS https://github.com/contourpy/contourpy/actions/runs/5338853882/jobs/9676798106
• Windows https://github.com/contourpy/contourpy/actions/runs/5338853882/jobs/9676798261

[tacaswell]

I am going to close this as we are able to install Matplotlib on the CPython betas (and the CPython main branch) in other contexts (locally for me, the fedora py312 rebuilds, the CI from contourpy).

This looks like build issues on windows, specifically in the binary-wheel packaging ecosystem, for some of our upstream dependencies.

[cclauss]

Why close the issue if python3.12 -m pip install matplotlib fails?!?

Why attach this issue to the v3.7.2 milestone and close that milestone if python3.12 -m pip install matplotlib==3.7.2 fails?!?

[jklymak]

@cclauss python 3.12 is not released yet. If you want to install Matplotlib with 3.12, you will have to do so manually (https://matplotlib.org/stable/users/installing/index.html#installing-from-source should work).

[oscargus]

I can also add that we now do run 3.12-tests as part of the regular CI, so we are quite confident that it is possible to install the main branch on 3.12 from source. There are also nightly wheels published for 3.12 that can be installed with pip: https://matplotlib.org/stable/users/installing/index.html#installing-a-nightly-build

[cclauss]

I was looking for something I could put in a requirements.txt file so I can work on other dependencies.

% docker run --rm -it python:3.12.0rc2 bash

Inside the Docker container...

root@9b993550be2e:/# python --version
Python 3.12.0rc2
root@9b993550be2e:/# python -m pip install numpy==v1.26.0rc1
# Success. ;-)
root@9b993550be2e:/# python -m pip install -e 'git+https://github.com/matplotlib/matplotlib@main#egg=matplotlib'
# Trys to import an older numpy -- Failure. :-(

[jklymak]

If you are having troubles installing release candidate based packages, please discuss at https://discourse.matplotlib.org. We don't officially support our install toolchain for pre-release candidates, so this isn't appropriate for our bug tracker. However, if you go to discourse, someone may be able to point you in the right direction.

[tacaswell]

This is an issue with pip defaulting to build things in a temporary environment. The new environment will not have numpy installed to it will try to build a release version of numpy which will fail.

Use pip install --no-build-isolation ... and it should work.

---

If you are interested in building default branches of everything please have a look at https://github.com/tacaswell/build_the_world

For a long discussion of the general issues with the pip-wheel ecosystem (that I think this is one aspect of) see https://pypackaging-native.github.io