pip install fails under Python 3.9 if numpy isn't already installed

[MattiasOlla]

Description

Installing scikit-learn via pip on linux under Python 3.9 fails if numpy is not already installed. This makes it impossible (or at least much harder) to use in requirements.txt files when creating virtual environments or docker containers.

Way to reproduce

Start with a clean virtual envoronment

python -m venv .env
. .env/bin/activate
pip install scikit-image

Results in the following error:

    ERROR: Command errored out with exit status 1:
     command: FOLDER/.env/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rvixazx6/scikit-image/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rvixazx6/scikit-image/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-5_ug79dv
         cwd: /tmp/pip-install-rvixazx6/scikit-image/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rvixazx6/scikit-image/setup.py", line 243, in <module>
        'build_ext': openmp_build_ext(),
      File "/tmp/pip-install-rvixazx6/scikit-image/setup.py", line 71, in openmp_build_ext
        from numpy.distutils.command.build_ext import build_ext
    ModuleNotFoundError: No module named 'numpy'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I have also tried pinning the version to 0.17.1 and 0.16.2, which doesn't work either. Running pip install numpy scikit-learn also fails, to make it succeed, they must be run separately: pip install numpy && pip install scikit-image.

Version information

# Paste the output of the following python commands
import sys; print(sys.version)
import platform; print(platform.platform())

3.9.0 (default, Nov  3 2020, 13:19:45) 
[GCC 7.5.0]
Linux-4.15.0-122-generic-x86_64-with-glibc2.27

[emmanuelle]

Thanks for reporting this. I believe this problem (using numpy in our setup.py before numpy is installed) is a bug which has been fixed since 0.17.2 (in #4920, see in particular https://github.com/scikit-image/scikit-image/blob/master/setup.py#L167).
Also we don't have a wheel for python 3.9, hence the problem appears.

@scikit-image/core should we backport #4920 (or part of it) to 0.17.3?

[hmaarrfk]

IMO: I don't think a 0.17.3 will happen. I think release a 0.18 package will be able to solve this particular challenge.

[emmanuelle]

We had discussed with @jni whether to release 0.17.3 just before 0.18 but I'm not 100% sure whether this will happen. 0.18 would also work of course.

[cowlinator]

@emmanuelle ,

I don't understand how #4920 is supposed to resolve this issue.

Everything in setup.py below line 14

scikit-image/setup.py

Line 14 in 79cb22a

from numpy.distutils.command.build_ext import build_ext

seems irrelevant, because setup.py has already crashed due to not being able to find numpy by that point.

#4920 put numpy into the requirements in pyproject.toml.

Does this mean that numpy needs to be installed before attempting to install scikit-image?

If so, this still makes it impossible to install numpy and scikit-image together in a single requirements.txt file.

This pre-installation requirement is not documented on the readme.

[stefanv]

NumPy is a build dependency of scikit-image, so you cannot run scikit-image's setup.py without it. However, if you install scikit-image from wheel (as you typically do when specifying requirements.txt), this should pose no problem.

The build requirements are documented here: https://scikit-image.org/docs/dev/install.html#full-requirements-list

[hmaarrfk]

python 3.9 was release Oct 2020. I somewhat remember this as being related to an early adopter getting stuck compiling bumpy from sdist.

I believe that the mentioned PRs (and subsequent releases) resolved the issue today.

I do not anticipate the same issue for 3.10.

though we should likely try to start releasing wheels for it.

[MattiasOlla]

@hmaarrfk I can indeed confirm that I was trying to adopt Python 3.9 quite early (foolishly early, some might say).

[stefanv]

We could also fail more gracefully in setup.py when numpy.distutils cannot be imported.

[addisonklinke]

@stefanv NumPy is a build dependency of scikit-image, so you cannot run scikit-image's setup.py without it. However, if you install scikit-image from wheel

@emmanuelle Says there are no wheels for Python 3.9 and the PyPI page says scikit-learn requires Python >= 3.8. Given this, I tried scikit-image==0.19.2 in a venv with Python 3.8.12 hoping that there would be a wheel so I could avoid manually installing numpy first. Unfortunately there is no wheel 😞 - which Python versions actually have wheels then?

Side note: the docs linked by @stefanv seem to be out-of-date since they say Python >= 3.6

EDIT: noted the specific version of scikit-learn that was attempted

[hmaarrfk]

I'm sorry for not closing this issue before hand. We really should have 1 year ago. It is just hard to do so without getting confirmation from the end user that their issues have been resolved.

I feel like your challenges in, 2022, are quite different than those in 2020.

We would be happy to help but we do require more information from you. We request it when you try to open a new issue.

It really helps!

To answer your specific questions:

1. The PyPi page is likely the most updated. We do require Python 3.8 https://pypi.org/project/scikit-image/#files

2. On that page, you will find that we have many wheels for python 3.8
   

3. The docs may be out of date slightly. Development is always a moving target. A PR to update those to reflect the new normal would be appreciated.

When you open a new issue, you will have a chance to tell us about your operaitng system and hardware. those are really important to know when you try to install software.

[addisonklinke]

Ah good to know the "files" page on PyPI, thanks! I had been trying scikit-image 0.19.2 with Python 3.8.12, and looking at the list the latest supported is Python 3.7, so that makes sense why it couldn't find any

Ultimately, 0.19.2 is extremely outdated (I only tried it because Yolo V7 has it pinned in their requirements.txt). I ended up using the latest 1.1.2 which has many options available for wheels

[alexdesiqueira]

@addisonklinke I think you've mistaken us (scikit-image) for our friends at scikit-learn... 😄