pyBigWig does not install through install_requires for python package

[msbentsen]

Hi,

I want to set pyBigWig as an installation dependency for my own python package using install_requires in setup.py, but I am having trouble during the installation procedure.

My minimal setup.py looks like this:

from setuptools import setup
setup(name='mypackage',
      install_requires=["pyBigWig"])

Upon python setup.py install I get the following messages/errors:

Processing dependencies for mypackage==0.0.0
Searching for pyBigWig
Reading https://pypi.org/simple/pyBigWig/
Downloading https://files.pythonhosted.org/packages/5c/9c/2e01d04b6fd31c1e09537c8f51410123c087da4974bdb0fcdb7ec25dab62/pyBigWig-0.3.13.tar.gz#sha256=c65cb9ccd78a4ede00c3209579173e4f7e67598daf6dc079676a0e8982163f5b
Best match: pyBigWig 0.3.13
Processing pyBigWig-0.3.13.tar.gz
Writing /tmp/easy_install-swqddsi0/pyBigWig-0.3.13/setup.cfg
Running pyBigWig-0.3.13/setup.py -q bdist_egg --dist-dir /tmp/easy_install-swqddsi0/pyBigWig-0.3.13/egg-dist-tmp-r3gwo7hc
/home/mbentse/.conda/envs/mypackage_env/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'classifier'
  warnings.warn(msg)
zip_safe flag not set; analyzing archive contents...
__pycache__.pyBigWig.cpython-37: module references __file__
pyBigWigTest.__pycache__.test.cpython-37: module references __file__
No eggs found in /tmp/easy_install-swqddsi0/pyBigWig-0.3.13/egg-dist-tmp-r3gwo7hc (setup script problem?)
error: The 'pyBigWig' distribution was not found and is required by mypackage

Strangely pip install pybigwig works, but I would love for it to recognize the package automatically so that I don't have to handle the exception when importing pybigwig later on.

Hope you can help out with a solution!

[dpryan79]

Is there a reason you're not using pip install?

[msbentsen]

Mainly just to limit the amount of setup commands needed to install the full package with all dependencies. I also eventually want to push my package as a pypi package, and as far as I understand, setup.py will define the dependencies (please correct me if I am wrong?) and I am not sure if "pip install" will then correctly handle installation of pybigwig during setup.

It is essentially a minor problem since I can also catch it with an exception and write a nice error message if trying to build the package without pybigwig or trying to import pybigwig without being installed - but if possible, I would of course rather like it to be handled automatically.

[dpryan79]

All I can say is that this works fine with deepTools using both pip install and python setup.py install, though usually we're using conda to ensure the dependencies are all there to begin with.

[dpryan79]

BTW, pip is just using whatever is in setup.py as far as I know.

[msbentsen]

For me, "python setup.py install" for deepTools also does not install dependencies (runs through without error but dependencies numpy/pybigwig etc. are not available), but I agree that conda is probably the way to go. I will find another solution to my setup. Thanks for the quick answers!

[barrel0luck]

This is the closest issue I found to my problem. I'm unable to install deeptools via pip:
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-qVY26u/subprocess32/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-mdnPME/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-qVY26u/subprocess32/

or pip3
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-bidkekxc/pyBigWig/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-nzki5ppf/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-bidkekxc/pyBigWig/

The issue seems to be with instlaling pyBigWig.

Installing via git works (as in there seem to be no errors), but the packages (numpy, pyBigWig, etc) do not get installed and so deeptools does not work.

I really don't like installing via conda, as it's difficult to use via R/ Rstudio... I've never got that working...
(I'm on Fedora 30).

[barrel0luck]

My problem was solved after installing python2-devel. pip works perfectly after that. Might help others.

[dpryan79]

That's a very good hint, the headers in particular are required for C extensions to compile properly with python.

[barrel0luck]

Yeah you are right cause even this was there in the error message:

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
    _posixsubprocess.c:16:10: fatal error: Python.h: No such file or directory
       16 | #include "Python.h"
          |          ^~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-PvGCIU/subprocess32/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-MWHpBe/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-PvGCIU/subprocess32/

This must've been a common problem that you've dealt with before. Apologies for adding noise here.