Update setuptools 63.4.3, setuptools_wheel 65.x, tomlkit 0.11.4, importlib_resources 5.9.0, importlib_metadata 4.12.0, distlib 0.3.6, hatchling 1.10.0, pathspec 0.10.1

[mkoeppe]

Numpy's distutils-based build system needs setuptools < 64. We upgrade to the latest suitable version, 63.4.3, which will be used in non-PEP-517 builds. https://setuptools.pypa.io/en/latest/history.html#v63-4-3

For PEP 517 builds, we provide the latest setuptools (65.x) as a wheel, which will be used in the isolated build environments of PEP 517 packages. https://setuptools.pypa.io/en/latest/history.html#v65-3-0

The update to 65.3 brings PEP 660 editable installs.
Here on this ticket, we disable them. The follow-up ticket #34209 will switch editable builds to this new technology.

We also update a few more build system related packages.

• tomlkit: Update to 0.11.4
• importlib_resources: Update to 5.9.0
• importlib_metadata: Update to 4.12.0
• distlib: Update to 0.3.6
• hatchling: Update to 1.10.0
• pathspec: Update to 0.10.1

Depends on #34442

CC: @kiwifb @antonio-rojas @saraedum @jhpalmieri

Component: packages: standard

Author: Matthias Koeppe

Branch/Commit: 664e98c

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/34344

[mkoeppe]

Branch: u/mkoeppe/update_setuptools_to_64_0_0

[mkoeppe]

New commits:

c06ecec	build/pkgs/setuptools: Update to 64.0.0
a292013	build/bin/sage-dist-helpers (sdh_pip_editable_install), build/pkgs/sagelib/spkg-install: Do not use PEP 660 editable wheels

[mkoeppe]

Commit: a292013

[mkoeppe]

comment:3

This version of setuptools breaks the build of our numpy-1.22.4 package

    File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 157, in __getattr__
      return _build_ext.build_ext.__getattr__(self, name)
    File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 105, in __getattr__
      raise AttributeError(attr)
  AttributeError: fcompiler
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1

[mkoeppe]

Dependencies: #34110

[mkoeppe]

comment:5

Actually it needs to be SETUPTOOLS_ENABLE_FEATURES (plural) pypa/setuptools#3538

[sagetrac-git]

Changed commit from a292013 to 08f18eb

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f7e7554	build/pkgs/setuptools: Update to 65.3.0
08f18eb	Use SETUPTOOLS_ENABLE_FEATURES, not SETUPTOOLS_ENABLE_FEATURE

[sagetrac-git]

Changed commit from 08f18eb to 71f8681

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

71f8681

build/pkgs/setuptools/distros/conda.txt: Use setuptools < 64

[mkoeppe]

comment:8

The updated setuptools still causes the same build failure with our numpy 1.22.4.

[mkoeppe]

Changed dependencies from #34110 to #34110, #34442

[mkoeppe]

Work Issues: Revert #34442

[mkoeppe]

Changed work issues from Revert #34442 to Revert #34442's install-requires (but keep pin in conda.txt)

[mkoeppe]

comment:12

The upgrade to numpy 1.23.3 (#34110) does not help.

In fact, upstream numpy is moving away from numpy.distutils and does not intend to keep it working with newer setuptools - numpy/numpy#22154

Breaking the numpy build system has been raised as pypa/setuptools#3549 - no solution as of 2022-09-23

[mkoeppe]

Changed dependencies from #34110, #34442 to #34442

[sagetrac-git]

Changed commit from 71f8681 to 34dee19

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

fdf9099	build/bin/sage-dist-helpers (sdh_pip_editable_install), build/pkgs/sagelib/spkg-install: Do not use PEP 660 editable wheels
94f0f76	build/pkgs/setuptools[_wheel]: Update to 63.4.3, 65.3.0
34dee19	Revert "build/pkgs/setuptools/install-requires.txt: Set upper bound"

[mkoeppe]

Changed work issues from Revert #34442's install-requires (but keep pin in conda.txt) to none

[sagetrac-git]

Changed commit from 34dee19 to ab0e195

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

87f2300	build/pkgs/hatchling: Update to 1.10.0
045ee7d	build/pkgs/distlib: Update to 0.3.6
7529bc4	build/pkgs/importlib_metadata: Update to 4.12.0
2fa5306	build/pkgs/importlib_resources: Update to 5.9.0
ab0e195	build/pkgs/tomlkit: Update to 0.11.4

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c7d9032

build/pkgs/pathspec: Update to 0.10.1

[sagetrac-git]

Changed commit from ab0e195 to c7d9032

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

bda724d

build/pkgs/setuptools_wheel: Update to 65.4.0

[sagetrac-git]

Changed commit from c7d9032 to bda724d

[jhpalmieri]

comment:23

This almost looks like a straightforward upgrade. My only questions are about setuptools vs. setuptools_wheel: how does numpy (and maybe that's the only relevant package) know not to use setuptools_wheel? And could there be there an issue with having mismatched versions of setuptools and setuptools_wheel?

[mkoeppe]

comment:24

Since #33789, we use build isolation for all Python packages that use sdh_pip_install - except for those that we invoke with --no-use-pep517 or --no-build-isolation.
When build isolation is in use, it does not matter what is installed in site-packages, as it creates a fresh build environment using the wheels in venv/var/lib/sage/wheels/; this includes the setuptools wheel, which is provided by our setuptools_wheel SPKG.
The versions don't need to be synchronized - a setuptools-based package uses exactly one of the two.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

664e98c

build/pkgs/distlib/checksums.ini: switch to tar.gz

[sagetrac-git]

Changed commit from bda724d to 664e98c

[jhpalmieri]

comment:26

Okay, let's merge it.

[jhpalmieri]

Reviewer: John Palmieri

[mkoeppe]

comment:27

Thanks!

[vbraun]

Changed branch from u/mkoeppe/update_setuptools_to_64_0_0 to 664e98c