BUG - unable to install celer in an empty python virtual environment

[Badr-MOUFAD]

minimal steps to reproduce,

1. cd to a new directory
2. create a python environment python -m venv venv
3. install celer pip install -U celer

Error logs

(click to expend)

  import numpy.distutils.command.sdist
  Traceback (most recent call last):
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 251, in generate_cython
  ModuleNotFoundError: No module named 'Cython'
 
  The above exception was the direct cause of the following exception:
 
  Traceback (most recent call last):
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 154, in save_modules
      yield saved
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context
      yield
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 250, in run_setup
      _execfile(setup_script, ns)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 45, in _execfile
      exec(code, globals, locals)
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 493, in <module>
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 475, in setup_package
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 258, in generate_cython
  OSError: Cython needs to be installed in Python as a module
 
  During handling of the above exception, another exception occurred:
 
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "C:\Users\HP\AppData\Local\Temp\pip-req-build-87kkcnrt\setup.py", line 4, in <module>
      dist.Distribution().fetch_build_eggs(['numpy>=1.12'])
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\dist.py", line 716, in fetch_build_eggs
      resolved_dists = pkg_resources.working_set.resolve(
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 780, in resolve
      dist = best[req.key] = env.best_match(
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 1065, in best_match
      return self.obtain(req, installer)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\pkg_resources\__init__.py", line 1077, in obtain
      return installer(requirement)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\dist.py", line 786, in fetch_build_egg
      return cmd.easy_install(req)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 679, in easy_install      
      return self.install_item(spec, dist.location, tmpdir, deps)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 705, in install_item      
      dists = self.install_eggs(spec, download, tmpdir)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 890, in install_eggs      
      return self.build_and_install(setup_script, setup_base)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 1158, in 
   build_and_install          self.run_setup(setup_script, setup_base, args)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\command\easy_install.py", line 1144, in run_setup        
      run_setup(setup_script, args)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 253, in run_setup
      raise
    File "C:\Users\HP\AppData\Local\Programs\Python\Python38-32\lib\contextlib.py", line 131, in __exit__
      self.gen.throw(type, value, traceback)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context
      yield
    File "C:\Users\HP\AppData\Local\Programs\Python\Python38-32\lib\contextlib.py", line 131, in __exit__
      self.gen.throw(type, value, traceback)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 166, in save_modules
      saved_exc.resume()
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 141, in resume
      six.reraise(type, exc, self._tb)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\_vendor\six.py", line 685, in reraise
      raise value.with_traceback(tb)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 154, in save_modules
      yield saved
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 195, in setup_context
      yield
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 250, in run_setup
      _execfile(setup_script, ns)
    File "C:\Users\HP\Desktop\install-celer\venv\lib\site-packages\setuptools\sandbox.py", line 45, in _execfile
      exec(code, globals, locals)
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 493, in <module>
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 475, in setup_package
    File "C:\Users\HP\AppData\Local\Temp\easy_install-unrz350b\numpy-1.23.0rc3\setup.py", line 258, in generate_cython
  OSError: Cython needs to be installed in Python as a module
  [end of output]

 note: This error originates from a subprocess, and is likely not a problem with pip.
 error: metadata-generation-failed

  × Encountered error while generating package metadata.
  ╰─> See above for output.

Additional comments

It seems that the error is related to fetching NumPy builds

celer/setup.py

Line 4 in bd11a44

dist.Distribution().fetch_build_eggs(['numpy>=1.12'])

Also, a portion of the error, namely

OSError: Cython needs to be installed in Python as a module
      [end of output]

is related to https://github.com/numpy/numpy/blob/705244444526804860e9f8d52459e2ca5c255366/setup.py#L257-L258

[Badr-MOUFAD]

@mathurinm could you pls verify whether this bug is OS related?

[mathurinm]

I use conda envs not virtual envs, can you give me steps to reproduce ?

Is it related to the wheel buidling we released in 0.7 ? Does pip install celer --no-binary work? And with version 6.1 ?

If it fails in an empty env it will fail in CI's of benchmarks were celer is a requirement, we can check that too

[Badr-MOUFAD]

I use conda envs not virtual envs, can you give me steps to reproduce ?

The bug doesn't occur in conda environment.

Is it related to the wheel buidling we released in 0.7 ?

No, we also get the same bug for previous versions.

[Badr-MOUFAD]

This was solved after releasing wheels (#274) for Windows, macOS, and Linux.