ENH: Support python 3.12 #3305

connortann · 2023-10-04T08:09:26Z

Problem Description

Release wheels for python 3.12.

Work-in-progress PR: #3414

Core dependencies:

numba: Python 3.12 numba/numba#9197

Test dependencies:

catboost (awaiting release 1.2.3): catboost 1.2.2 installation error in python 3.12 catboost/catboost#2520
tensorflow: Support Python 3.12 tensorflow/tensorflow#62003
ngboost: Add support for python 3.12 stanfordmlgroup/ngboost#345

The text was updated successfully, but these errors were encountered:

connortann · 2023-11-29T21:14:50Z

Numba supports 3.12 on the main branch, so in terms of core dependencies we're now just awaiting the next release:

we expect the upcoming Numba 0.59 to support 3.12.

We will also need to consider which of our optional dependencies support 3.12

xgboost: ?
tensorflow: Support Python 3.12 tensorflow/tensorflow#62003
pytorch: Pytorch for Python 3.12 not available pytorch/pytorch#110436

RNarayan73 · 2024-02-02T19:09:59Z

Numba supports 3.12 on the main branch, so in terms of core dependencies we're now just awaiting the next release:

we expect the upcoming Numba 0.59 to support 3.12.

We will also need to consider which of our optional dependencies support 3.12

xgboost: ?

tensorflow: Support Python 3.12 tensorflow/tensorflow#62003

pytorch: Pytorch for Python 3.12 not available pytorch/pytorch#110436

I have managed to install xgboost in a Python 3.12 environment, however, haven't got the environment functioning yet due to other dependencies

RNarayan73 · 2024-02-02T19:29:16Z

Following error occurs when trying to install shap in a Python 3.12 environment:

(skl312) PS C:\Users\naray\Python\venvs\skl312> pip install shap

Collecting shap
  Using cached shap-0.44.1.tar.gz (1.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (1.26.3)
Requirement already satisfied: scipy in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (1.12.0)
Requirement already satisfied: scikit-learn in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (1.4.0)
Requirement already satisfied: pandas in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (2.2.0)
Requirement already satisfied: tqdm>=4.27.0 in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (4.66.1)
Requirement already satisfied: packaging>20.9 in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (23.2)
Requirement already satisfied: slicer==0.0.7 in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (0.0.7)
Requirement already satisfied: numba in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (0.59.0)
Requirement already satisfied: cloudpickle in c:\users\naray\python\venvs\skl312\lib\site-packages (from shap) (3.0.0)
Requirement already satisfied: colorama in c:\users\naray\python\venvs\skl312\lib\site-packages (from tqdm>=4.27.0->shap) (0.4.6)
Requirement already satisfied: llvmlite<0.43,>=0.42.0dev0 in c:\users\naray\python\venvs\skl312\lib\site-packages (from numba->shap) (0.42.0)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\naray\python\venvs\skl312\lib\site-packages (from pandas->shap) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in c:\users\naray\python\venvs\skl312\lib\site-packages (from pandas->shap) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in c:\users\naray\python\venvs\skl312\lib\site-packages (from pandas->shap) (2023.4)
Requirement already satisfied: joblib>=1.2.0 in c:\users\naray\python\venvs\skl312\lib\site-packages (from scikit-learn->shap) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\users\naray\python\venvs\skl312\lib\site-packages (from scikit-learn->shap) (3.2.0)
Requirement already satisfied: six>=1.5 in c:\users\naray\python\venvs\skl312\lib\site-packages (from python-dateutil>=2.8.2->pandas->shap) (1.16.0)
Building wheels for collected packages: shap
  Building wheel for shap (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for shap (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [176 lines of output]
      Attempting to build SHAP: with_binary=True, with_cuda=True (Attempt 1)
      The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME to enable CUDA.
      Exception occurred during setup, Error building cuda module: TypeError('cannot unpack non-iterable NoneType object')
      WARNING: Could not compile cuda extensions.
      Retrying SHAP build without cuda extension...
      Attempting to build SHAP: with_binary=True, with_cuda=False (Attempt 2)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-312
      creating build\lib.win-amd64-cpython-312\shap
      copying shap\datasets.py -> build\lib.win-amd64-cpython-312\shap
      copying shap\links.py -> build\lib.win-amd64-cpython-312\shap
      copying shap\_explanation.py -> build\lib.win-amd64-cpython-312\shap
      copying shap\_serializable.py -> build\lib.win-amd64-cpython-312\shap
      copying shap\_version.py -> build\lib.win-amd64-cpython-312\shap
      copying shap\__init__.py -> build\lib.win-amd64-cpython-312\shap
      creating build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\pytree.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\tf_utils.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_additive.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_exact.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_explainer.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_gpu_tree.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_gradient.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_kernel.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_linear.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_partition.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_permutation.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_sampling.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\_tree.py -> build\lib.win-amd64-cpython-312\shap\explainers
      copying shap\explainers\__init__.py -> build\lib.win-amd64-cpython-312\shap\explainers
      creating build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_coefficient.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_lime.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_maple.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_random.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_treegain.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\_ubjson.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      copying shap\explainers\other\__init__.py -> build\lib.win-amd64-cpython-312\shap\explainers\other
      creating build\lib.win-amd64-cpython-312\shap\explainers\_deep
      copying shap\explainers\_deep\deep_pytorch.py -> build\lib.win-amd64-cpython-312\shap\explainers\_deep
      copying shap\explainers\_deep\deep_tf.py -> build\lib.win-amd64-cpython-312\shap\explainers\_deep
      copying shap\explainers\_deep\deep_utils.py -> build\lib.win-amd64-cpython-312\shap\explainers\_deep
      copying shap\explainers\_deep\__init__.py -> build\lib.win-amd64-cpython-312\shap\explainers\_deep
      creating build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_bar.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_beeswarm.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_benchmark.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_decision.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_embedding.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_force.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_force_matplotlib.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_group_difference.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_heatmap.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_image.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_labels.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_monitoring.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_partial_dependence.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_scatter.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_text.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_utils.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_violin.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\_waterfall.py -> build\lib.win-amd64-cpython-312\shap\plots
      copying shap\plots\__init__.py -> build\lib.win-amd64-cpython-312\shap\plots
      creating build\lib.win-amd64-cpython-312\shap\plots\colors
      copying shap\plots\colors\_colorconv.py -> build\lib.win-amd64-cpython-312\shap\plots\colors
      copying shap\plots\colors\_colors.py -> build\lib.win-amd64-cpython-312\shap\plots\colors
      copying shap\plots\colors\__init__.py -> build\lib.win-amd64-cpython-312\shap\plots\colors
      creating build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\experiments.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\framework.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\measures.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\methods.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\metrics.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\models.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\plots.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\_compute.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\_explanation_error.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\_result.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\_sequential.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      copying shap\benchmark\__init__.py -> build\lib.win-amd64-cpython-312\shap\benchmark
      creating build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_composite.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_fixed.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_fixed_composite.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_image.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_masker.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_output_composite.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_tabular.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\_text.py -> build\lib.win-amd64-cpython-312\shap\maskers
      copying shap\maskers\__init__.py -> build\lib.win-amd64-cpython-312\shap\maskers
      creating build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\image.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\transformers.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_clustering.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_exceptions.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_general.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_keras.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_legacy.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_masked_model.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\_show_progress.py -> build\lib.win-amd64-cpython-312\shap\utils
      copying shap\utils\__init__.py -> build\lib.win-amd64-cpython-312\shap\utils
      creating build\lib.win-amd64-cpython-312\shap\actions
      copying shap\actions\_action.py -> build\lib.win-amd64-cpython-312\shap\actions
      copying shap\actions\_optimizer.py -> build\lib.win-amd64-cpython-312\shap\actions
      copying shap\actions\__init__.py -> build\lib.win-amd64-cpython-312\shap\actions
      creating build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\_model.py -> build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\_teacher_forcing.py -> build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\_text_generation.py -> build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\_topk_lm.py -> build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\_transformers_pipeline.py -> build\lib.win-amd64-cpython-312\shap\models
      copying shap\models\__init__.py -> build\lib.win-amd64-cpython-312\shap\models
      running egg_info
      writing shap.egg-info\PKG-INFO
      writing dependency_links to shap.egg-info\dependency_links.txt
      writing requirements to shap.egg-info\requires.txt
      writing top-level names to shap.egg-info\top_level.txt
      reading manifest file 'shap.egg-info\SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file 'shap.egg-info\SOURCES.txt'
      C:\Users\naray\AppData\Local\Temp\pip-build-env-dwgall6f\overlay\Lib\site-packages\setuptools\command\build_py.py:207: _Warning: Package 'shap.cext' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'shap.cext' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'shap.cext' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'shap.cext' to be distributed and are
              already explicitly excluding 'shap.cext' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      creating build\lib.win-amd64-cpython-312\shap\cext
      copying shap\cext\_cext.cc -> build\lib.win-amd64-cpython-312\shap\cext
      copying shap\cext\_cext_gpu.cc -> build\lib.win-amd64-cpython-312\shap\cext
      copying shap\cext\_cext_gpu.cu -> build\lib.win-amd64-cpython-312\shap\cext
      copying shap\cext\gpu_treeshap.h -> build\lib.win-amd64-cpython-312\shap\cext
      copying shap\cext\tree_shap.h -> build\lib.win-amd64-cpython-312\shap\cext
      creating build\lib.win-amd64-cpython-312\shap\plots\resources
      copying shap\plots\resources\bundle.js -> build\lib.win-amd64-cpython-312\shap\plots\resources
      copying shap\plots\resources\logoSmallGray.png -> build\lib.win-amd64-cpython-312\shap\plots\resources
      running build_ext
      building 'shap._cext' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for shap
Failed to build shap
ERROR: Could not build wheels for shap, which is required to install pyproject.toml-based projects

shap wheel needs to be updated for Python 3.12

Thanks
Narayan

connortann added enhancement Indicates new feature requests dependencies Pull requests that update a dependency file labels Oct 4, 2023

connortann mentioned this issue Oct 4, 2023

[Meta issue] Release 0.43.0 #3289

Closed

7 tasks

radlfabs mentioned this issue Oct 29, 2023

Python 3.12 radlfabs/flexcv#4

Open

connortann mentioned this issue Nov 29, 2023

Add support for python 3.12, drop support for 3.8 #3414

Merged

connortann mentioned this issue Jan 24, 2024

shap installation failed; python version- 3.12.1 #3468

Closed

connortann mentioned this issue Feb 5, 2024

MAINT: Restore ngboost tests for py311 #3483

Merged

connortann closed this as completed in #3414 Feb 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Support python 3.12 #3305

ENH: Support python 3.12 #3305

connortann commented Oct 4, 2023 •

edited

connortann commented Nov 29, 2023 •

edited

RNarayan73 commented Feb 2, 2024 •

edited

RNarayan73 commented Feb 2, 2024

ENH: Support python 3.12 #3305

ENH: Support python 3.12 #3305

Comments

connortann commented Oct 4, 2023 • edited

Problem Description

connortann commented Nov 29, 2023 • edited

RNarayan73 commented Feb 2, 2024 • edited

RNarayan73 commented Feb 2, 2024

connortann commented Oct 4, 2023 •

edited

connortann commented Nov 29, 2023 •

edited

RNarayan73 commented Feb 2, 2024 •

edited