Use of numpy.distutils in setup.py causes errors if installing in clean virtualenv

[jcohen02]

Describe the bug
When attempting to install Solcore in a clean virtualenv (or any Python environment that does not currently have numpy installed) using python setup.py install results in an error since there is an attempt to import the version of distutils from wihtin numpy prior to numpy being installed!

To Reproduce
Clone solcore, create and activate a virtualenv and then attempt to run the install using setup.py. This example is based on a Mac/Linux platform with Python >=3.7 and the virtualenv Python package installed:

git clone git@github.com:qpv-research-group/solcore5.git
cd solcore5
virtualenv env
source env/bin/activate
python setup.py install

This immediately results in the error:

  File "setup.py", line 2, in <module>
    from numpy.distutils.core import Extension, setup
ModuleNotFoundError: No module named 'numpy'

Expected behavior
The install should proceed and complete successfully.

Additional context
This is a result of attempting to use the enhanced distutils from within numpy prior to numpy being installed.
It looks like this version of distutils is required to support the building of the native PDD library from the Fortran source when using the --with_pdd option.

A simple workaround would be to import only setup from distutils.core and then import Extension from numpy.distutils.core within the if "--with_pdd" in sys.argv: block. This would still require someone wanting to use the PDD module to install numpy manually first but it would at least prevent an error when trying to run the basic install.
I assume there's a better solution to this which would presumably involve installing numpy inside the if "--with_pdd" in sys.argv: block if it's required.

[dalonsoa]

This is not a bug, but actually the expected behaviour. numpy is a pre-requisite for installing Solcore, as indicated in the documentation.

[jcohen02]

Thanks for highlighting this @dalonsoa. I wonder if it should still be possible to do an install from scratch in a clean environment using python setup.py install but, as you point out, installing numpy manually first will avoid the problem and since that's clear in the documentation, I'll close this issue.

[dalonsoa]

I wish it could be avoided, to be honest, but I think that the possible solutions are more convoluted than simply asking for numpy to be installed first. I wonder if, when having binary wheels, this could be avoided when installing Solcore from PyPI...

[jcohen02]

It seems this is a common issue that has been addressed (to some extent at least) in various repos, including scipy. See the discussion in the following for further info (not linking issues directly to avoid spamming these issues with reverse references):

Issue 4164 in scikit-learn: can't setup.py install without numpy
Issue 174 in scikit-hep: Bootstrap installation of numpy in setup.py if it is not already installed
How to Bootstrap numpy installation in setup.py

It looks like this could be addressed fairly straightforwardly using the approach taken in PR 292 in scikit-hep/root_numpy, however, if I understand correctly, the downside of this (as highlighted in one of the above links) is that then even running python setup.py --help results in numpy being installed if it isn't already. The solution to this is then a more substantial set of implementation to check the provided parameters and handle them individually where necessary as done in parse_setuppy_commands in setup.py of scipy.

I appreciate that this probably not classed as a critical requirement but might be something worth considering addressing going forward. Happy to look into resolving this if that would be helpful.

[Abelarm]

What about I take care of this?

[dalonsoa]

Absolutely! This has been bothering many people for a long time. If you can sort this out, it will be a great help!

[dalonsoa]

Solved by #202