ModuleNotFoundError: No module named 'numpy'

[Schwaggot]

I am trying to install classix 0.7.4 from a requirements file into a venv, but unfortunately I am getting an error about numpy not found, which kind of makes no sense to me, because it is installed in one of the steps above in version 1.22.4. I was hoping someone here could help me figure out what is going wrong.

System: Debian 11 (bullseye)
Python version: 3.9.2
CLASSIX version: 0.7.4
Cython: 0.29.32

Steps to reproduce:

$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install -r requirements.txt 
Collecting tqdm~=4.64.0
  Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting numpy~=1.22.4
  Using cached numpy-1.22.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting pandas~=1.4.2
  Using cached pandas-1.4.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Collecting seaborn~=0.11.2
  Using cached seaborn-0.11.2-py3-none-any.whl (292 kB)
Collecting matplotlib~=3.5.2
  Using cached matplotlib-3.5.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.2 MB)
Collecting tensorflow~=2.9.1
  Using cached tensorflow-2.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (511.8 MB)
Collecting scikit-learn~=1.1.1
  Using cached scikit_learn-1.1.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30.8 MB)
Collecting nfstream~=6.5.1
  Using cached nfstream-6.5.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting tabulate~=0.8.9
  Using cached tabulate-0.8.10-py3-none-any.whl (29 kB)
Collecting missingno~=0.5.1
  Using cached missingno-0.5.1-py3-none-any.whl (8.7 kB)
Collecting scipy~=1.8.1
  Using cached scipy-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.2 MB)
Collecting cython~=0.29.32
  Using cached Cython-0.29.32-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (2.0 MB)
Collecting scapy~=2.4.5
  Using cached scapy-2.4.5.tar.gz (1.1 MB)
Collecting zstandard~=0.18.0
  Using cached zstandard-0.18.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Collecting protobuf~=3.19.4
  Using cached protobuf-3.19.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting pyclustering~=0.10.1.2
  Using cached pyclustering-0.10.1.2.tar.gz (2.6 MB)
Collecting classixclustering~=0.7.4
  Using cached classixclustering-0.7.4.tar.gz (629 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/XXX/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-a5h4ms0x/classixclustering_5529ebbd0bef4c0489673327bfb2a134/setup.py'"'"'; __file__='"'"'/tmp/pip-install-a5h4ms0x/classixclustering_5529ebbd0bef4c0489673327bfb2a134/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-ldj6pnm6
         cwd: /tmp/pip-install-a5h4ms0x/classixclustering_5529ebbd0bef4c0489673327bfb2a134/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-a5h4ms0x/classixclustering_5529ebbd0bef4c0489673327bfb2a134/setup.py", line 1, in <module>
        import numpy
    ModuleNotFoundError: No module named 'numpy'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/4d/0f/5a17e5d8045195d1a112b143a8143fff86e558d7cbeacad886d1b93be6db/classixclustering-0.7.4.tar.gz#sha256=d0f72deccb40ca9eb14905bb1a0f41787a824446eebac5a67a7ae59ec4c65342 (from https://pypi.org/simple/classixclustering/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement classixclustering~=0.7.4
ERROR: No matching distribution found for classixclustering~=0.7.4

The content of the requirements.txt looks as follows:

tqdm~=4.64.0
numpy~=1.22.4
pandas~=1.4.2
seaborn~=0.11.2
matplotlib~=3.5.2
tensorflow~=2.9.1
scikit-learn~=1.1.1
nfstream~=6.5.1
tabulate~=0.8.9
missingno~=0.5.1
scipy~=1.8.1
cython~=0.29.32
scapy~=2.4.5
zstandard~=0.18.0
protobuf~=3.19.4
pyclustering~=0.10.1.2
classixclustering~=0.7.4
umap-learn~=0.5.3

Any help resolving this issue would be appreciated, thanks.

[chenxinye]

Hi @Schwaggot

Thanks for reporting this issue!
Seems like your machine does not have numpy installation, could you try to install it via

pip install numpy

Thanks,
Xinye

[Schwaggot]

Hi @chenxinye

thanks for the fast response, numpy is specified in the requirements file as

numpy~=1.22.4

and it is found during the pip install, you can see it in the logs:

Collecting numpy~=1.22.4
  Using cached numpy-1.22.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)

Or am I missing something?

Best,
Schwaggot

[Schwaggot]

Oh I see, running an

python3 -m pip install numpy==1.22.4

manually first resolves the error when trying to install install CLASSIX after that. But that seems very confusing, why is it necessary, I thought the idea of having the dependencies inside the python package would be exactly to avoid installing stuff by hand, it should figure out automatically what to install.

[chenxinye]

Hi @Schwaggot

Thank you very much for your feedback, it is indeed confusing at this point, we will have a look and give you answer as soon as possible. Also, if you have any idea on code or others, feel free to pull your request.

Cheers,
Xinye

[chenxinye]

@Schwaggot

We just fixed the issue you reported and release the new version 0.7.5.

Thank you very much!

[Schwaggot]

@chenxinye Thanks for the amazingly fast support!

It seems that the issue with numpy is fixed, but now the same happens with Cython...

Collecting classixclustering~=0.7.5
  Downloading classixclustering-0.7.5.tar.gz (627 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 627.1/627.1 kB 14.4 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-sgtkfiia/classixclustering_c1753dd87c1f473d8ee855d3935418f2/setup.py", line 4, in <module>
          from Cython import Distutils
      ModuleNotFoundError: No module named 'Cython'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[Schwaggot]

@chenxinye I might have a solution for it, it seems that adding Cython to build-time dependecies has worked:

fd964a8

I got the hint from here, similar issue: qutip/qutip#1246 (comment)
And also seems to be documented in Cython itself: https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#basic-setup-py

[Schwaggot]

Small update, I had a little issue with CLASSIX being compiled against a different version of numpy than being used later, so I included it into the pyproject.toml as well, it now looks as follows and works nicely:

[build-system]
requires = ["setuptools", "wheel", "Cython ~= 0.29.32", "numpy ~= 1.22.4"]

I suppose following that notation it would be possible to use the same >= etc. for numpy and Cython as in your setup.py.

[chenxinye]

Hi @Schwaggot,

Sorry for the inconvenience and late reply, please update classixclustering to 0.7,7 version.

We just fixed the setup for your new report issue, and now address all the dependency issues, and test it in the docker of native python and clean Debian environment. Thanks for your participation! Our appreciation is beyond words.

Cheers,
Xinye

[chenxinye]

Also, please let us know if you have any further questions, your feedback is really helpful!

[Schwaggot]

Works! Thank you so much for the fast support, keep up the good work!