BUG: trying to compile numpy 1.22.1 on results in failure

[aptalca]

Describe the issue:

Trying to compile numpy 1.22.1 on armhf (ubuntu focal) results in failure due to setuptools being too new (60.5.0) even though pip installs 5.9.20 as part of numpy install process. Numpy 1.22.0 compiles with no issues.

Reproduce the code example:

pip3 install -v numpy
(also tested on amd64 `pip install -v --no-binary numpy numpy`)

Error message:

Collecting numpy

  Downloading numpy-1.22.1.zip (11.4 MB)

  Installing build dependencies: started

  Running command /usr/bin/python3 /tmp/pip-standalone-pip-sn_sektr/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-aykwk_m_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --find-links https://wheel-index.linuxserver.io/ubuntu/ -- 'packaging==20.5; platform_machine=='"'"'arm64'"'"'' setuptools==59.2.0 wheel==0.37.0 'Cython>=0.29.24,<3.0'

  Looking in links: https://wheel-index.linuxserver.io/ubuntu/

  Ignoring packaging: markers 'platform_machine == "arm64"' don't match your environment

  Collecting setuptools==59.2.0

    Downloading setuptools-59.2.0-py3-none-any.whl (952 kB)

  Collecting wheel==0.37.0

    Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)

  Collecting Cython<3.0,>=0.29.24

    Downloading Cython-0.29.26-py2.py3-none-any.whl (983 kB)

  Installing collected packages: wheel, setuptools, Cython

  Successfully installed Cython-0.29.26 setuptools-59.2.0 wheel-0.37.0

  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

  Installing build dependencies: finished with status 'done'

  Getting requirements to build wheel: started

  Running command /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpv6q8qoeu

  Running from numpy source directory.

  Traceback (most recent call last):

    File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>

      main()

    File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main

      json_out['return_val'] = hook(**hook_input['kwargs'])

    File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel

      return hook(config_settings)

    File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel

      return self._get_build_requires(

    File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 143, in _get_build_requires

      self.run_setup()

    File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 267, in run_setup

      super(_BuildMetaLegacyBackend,

    File "/usr/local/lib/python3.8/dist-packages/setuptools/build_meta.py", line 158, in run_setup

      exec(compile(code, __file__, 'exec'), locals())

    File "setup.py", line 84, in <module>

      raise RuntimeError(

  RuntimeError: Setuptools version is '60.5.0', version < '60.0.0' is required. See pyproject.toml

  Getting requirements to build wheel: finished with status 'error'

NumPy/Python version information:

Python 3.8.10
pip 21.3.1
setuptools 60.5.0
wheel 0.37.1

[rossbar]

This seems like a likely environment issue - perhaps setuptools is not actually being updated, or is being picked up from elsewhere since it seems you're using the system python (note the WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv)

Consider using virtual environments to avoid conflicts with system packages, e.g. try

python -m venv my-env
source my-env/bin/activate.sh  # Or similar, depending on your shell
python -m pip install setuptools==59.2
python -m pip install numpy==1.22.1

[charris]

Yes, setuptools 60+ is known to fail, in 1.22.2 an error will be raised if it is detected during build. A proposed workaround is:
export SETUPTOOLS_USE_DISTUTILS=stdlib. I haven't tested that.

[aptalca]

Consider using virtual environments to avoid conflicts with system packages

Unfortunately this is part of a big ci/cd operation where we build and push wheels for various arches, cpython versions and glibc and musl to our wheel repo, which is used by all our docker images. We can't customize the environment for each package version so we have to rely on pip to pull in the necessary deps during build.
Here's our github repo: https://github.com/linuxserver/wheelie

Yes, setuptools 60+ is known to fail

The part that confuses me is that prior to building the numpy wheel, pip installs setuptools 59.2.0 per the message below:
Successfully installed Cython-0.29.26 setuptools-59.2.0 wheel-0.37.0
Not sure why the build doesn't detect and use it. I'll try in venv.

I tried SETUPTOOLS_USE_DISTUTILS=stdlib but it didn't seem to make a difference

Thanks

[charris]

I tried SETUPTOOLS_USE_DISTUTILS=stdlib but it didn't seem to make a difference

Thanks for trying that. From our point of view, numpy/distutils, which we and other projects rely on for C and Fortran support, will need to be replaced. SciPy is moving to meson and NumPy will follow when SciPy finishes. Others may move to cmake. Pure Pyhton projects should not have a problem. I expect there will be some pain during the transition.

[aptalca]

Thanks guys, you were right. Simply activating a venv prior to building the wheels solved this issue. I didn't have to downgrade setuptools first. I'll chalk it up to a pip issue with regards to the environment.