one go installation does not make extensions

[iurisegtovich]

hi

i tried installing the one go installation, but it did not make the fortran extension automatically.

after reading through the rest of the installation page i was able to make the extensions and load the package

i wonder if this is a bug in the installation script worth reporting

details below:

i installed via git clone and pip install .

then i was getting the error

from pwtools import pydos                                                                                                                                       
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-79e08d090104> in <module>
----> 1 from pwtools import pydos

~/pwtools/pwtools/pydos.py in <module>
     17 from scipy.fftpack import fft
     18 from scipy.signal import convolve, gaussian
---> 19 from pwtools import constants, _flib, num
     20 from pwtools.verbose import verbose
     21 from pwtools.signal import pad_zeros, welch, mirror

~/pwtools/pwtools/num.py in <module>
      8     bisplev, splev, splrep
      9 from scipy.integrate import simps, trapz
---> 10 from pwtools import _flib
     11 import warnings
     12 ##warnings.simplefilter('always')

ImportError: cannot import name '_flib'

the dependency test seemed fine

$ python pwtools/test/check_dependencies.py 
required packages:
    numpy ... ok
    scipy ... ok
    ase ... ok
    pyspglib ... ok
    CifFile ... ok
    h5py ... ok
    nose ... ok
    matplotlib ... ok
    sphinx ... ok
    numpydoc ... ok
optional executables:
    eos.x ... NOT FOUND

then i went into src and ran make following hints from
( http://elcorto.github.io/pwtools/written/install.html )

now it seems to work :)

[elcorto]

Hi, thanks for reporting this.

I tried the install in a fresh and empty venv and I can replicate that.

However what I think happens is that the extensions do get built, but when you run the import statement from within the repo /path/to/pwtools, the interpreter will try to import from the subdir /path/to/pwtools/pwtools since the first entry in sys.path is '', i.e. the current dir. It should work when you change to another dir. Then, the package is imported from the installed location, where the extension shared libs are located.

$ mkvirtualenv -p /usr/bin/python3 pwtools

(pwtools) $ pip list
Package       Version
------------- -------
pip           19.2.3
pkg-resources 0.0.0
setuptools    41.2.0
wheel         0.33.6

(pwtools) $ pip install numpy
(pwtools) $ pip install .

(pwtools) $ pip show pwtools
Name: pwtools
Version: 0.9.0
Summary: pre- and postprocessing of atomic calculations
Home-page: https://github.com/elcorto/pwtools
Author: Steve Schmerler
Author-email: git@elcorto.com
License: BSD 3-Clause
Location: /home/elcorto/.virtualenvs/pwtools/lib/python3.7/site-packages
Requires: numpy, numpydoc, ase, pyspglib, PyCifRW, scipy, matplotlib, h5py, nose
Required-by:

(pwtools) $ pwd
/home/elcorto/soft/git/pwtools

(pwtools) $ ls pwtools/*.so
<<nothing>>

(pwtools) $ ls /home/elcorto/.virtualenvs/pwtools/lib/python3.7/site-packages/pwtools/*.so                                                                                                                                   
/home/elcorto/.virtualenvs/pwtools/lib/python3.7/site-packages/pwtools/_dcd.so  /home/elcorto/.virtualenvs/pwtools/lib/python3.7/site-packages/pwtools/_flib.so

(pwtools) $ python -c "from pwtools import pydos"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/elcorto/soft/git/pwtools/pwtools/pydos.py", line 19, in <module>
    from pwtools import constants, _flib, num
  File "/home/elcorto/soft/git/pwtools/pwtools/num.py", line 10, in <module>
    from pwtools import _flib
ImportError: cannot import name '_flib' from 'pwtools' (/home/elcorto/soft/git/pwtools/pwtools/__init__.py)

(pwtools) $ cd ~
(pwtools) $ python -c "from pwtools import pydos"

(pwtools) $ python -c "import sys; print(sys.path)"
['', '/home/elcorto/.virtualenvs/pwtools/lib/python37.zip', '/home/elcorto/.virtualenvs/pwtools/lib/python3.7', '/home/elcorto/.virtualenvs/pwtools/lib/python3.7/lib-dynload', '/usr/lib/python3.7', '/home/elcorto/.virtualenvs/pwtools/lib/python3.7/site-packages']

This does not happen when you use the dev install pip install -e . b/c then, the extensions are located in /path/to/pwtools/pwtools.

(pwtools) $ cd /home/elcorto/soft/git/pwtools
(pwtools) $ pip uninstall pwtools
(pwtools) $ pip install -e .
(pwtools) $ pip show pwtools
Name: pwtools
Version: 0.9.0
Summary: pre- and postprocessing of atomic calculations
Home-page: https://github.com/elcorto/pwtools
Author: Steve Schmerler
Author-email: git@elcorto.com
License: BSD 3-Clause
Location: /home/elcorto/soft/git/pwtools
Requires: matplotlib, nose, numpy, scipy, h5py, ase, numpydoc, PyCifRW, pyspglib
Required-by:

(pwtools) $ python -c "from pwtools import pydos"

(pwtools) $ ls pwtools/*.so                                                                                                                                               
pwtools/_dcd.so  pwtools/_flib.so

But I found a flaw in the way we use setup.py. Since we build the extensions in a slightly quirky way, namely by calling make in setup.py before any package is installed, you need to install numpy first in order for f2py to be present :) This happens only when installing in a fresh venv w/o sharing the system site-packages which usually contain a numpy install already.

[elcorto]

The discovered install issue regarding missing numpy at build time is fixed.

Closing this one since (i) the reported issue was actually not a bug, judging from the information provided by the OP and (ii) the OP failed to provide feedback.