Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve installing via pip #121

Closed
JonnyJD opened this issue Feb 10, 2019 · 4 comments
Closed

improve installing via pip #121

JonnyJD opened this issue Feb 10, 2019 · 4 comments
Milestone

Comments

@JonnyJD
Copy link
Owner

JonnyJD commented Feb 10, 2019

isrcsubmit needs at least one non-python package (libdiscid, C library), so it can't be installed completely via pip.
However, when installing all python modules you will get a better error message like:

OSError: libdiscid.so.0: cannot open shared object file: No such file or directory

Pip then probably only installs into /usr/local/bin, which might or might not be in the $PATH.

This was probably the purpose of #116, even though was not mentioned in the subject.

@JonnyJD JonnyJD added this to the 2.0.2 milestone Feb 10, 2019
@JonnyJD
Copy link
Owner Author

JonnyJD commented Feb 10, 2019

I nearly forgot: the recommended install of the dependencies is either via the package manager (on linux), as that installs also libdiscid or to use one of the packages mentioned here (also including a libdiscid binary for Windows and Mac):
http://jonnyjd.github.io/musicbrainz-isrcsubmit/download

@eli-schwartz
Copy link

The current use of:

# we load isrcsubmit on setup
args["setup_requires"] = args["install_requires"],

Does not work. By the time setuptools has parsed this, the previous use of from isrcsubmit import __version__ has already run, and the setup.py has either successfully imported discid or aborted with a fatal error. This breaks the ability for setuptools to list and install the required dependencies.

There are three possible solutions:

  • Embrace https://www.python.org/dev/peps/pep-0518/ which can list buildsystem dependencies, it is supported by pip at least (but not plain setuptools)
  • refactor the imports to allow importing the version via a minimal import graph, probably requires having this in a separate version.py file usable by setup.py
  • Bump the version in two places on every release.

JonnyJD added a commit that referenced this issue Feb 22, 2019
so we don't have to import isrcsubmit.

This should enable setup.py to evaluate the dependencies
and pip to install the direct dependencies.

refs #121
JonnyJD added a commit that referenced this issue Feb 22, 2019
for setup.py and #121

So it looks like we did have an automatic way
to write the version in multiple places.
@JonnyJD
Copy link
Owner Author

JonnyJD commented Feb 22, 2019

This is basically implemented, but currently not active because I somehow can't deploy to PyPI:
Pypa/warehouse#5466

@JonnyJD JonnyJD modified the milestones: 2.0.2, 2.1.0 Feb 23, 2019
@JonnyJD
Copy link
Owner Author

JonnyJD commented Feb 23, 2019

After also fixing python-discid and testing with docker run -it ubuntu this seems to work now:

# pip3 install --no-cache-dir isrcsubmit
Collecting isrcsubmit
  Downloading https://files.pythonhosted.org/packages/a9/cf/bc0d85d52a0fb6735d02d8c6ccd7e852808e1d59370ebb741e22aa15558a/isrcsubmit-2.1.0.tar.gz (49kB)
    100% |################################| 51kB 837kB/s 
Collecting discid>=1.0.0 (from isrcsubmit)
  Downloading https://files.pythonhosted.org/packages/d5/fa/c8856ae3eb53393445d84589afbd49ded85527563a7c0457f4e967d5b7af/discid-1.2.0.tar.gz
Collecting musicbrainzngs>=0.4 (from isrcsubmit)
  Downloading https://files.pythonhosted.org/packages/63/cc/67ad422295750e2b9ee57c27370dc85d5b85af2454afe7077df6b93d5938/musicbrainzngs-0.6.tar.gz (111kB)
    100% |################################| 112kB 3.3MB/s 
Installing collected packages: discid, musicbrainzngs, isrcsubmit
  Running setup.py install for discid ... done
  Running setup.py install for musicbrainzngs ... done
  Running setup.py install for isrcsubmit ... done
Successfully installed discid-1.2.0 isrcsubmit-2.1.0 musicbrainzngs-0.6

Of course:

$ isrcsubmit.py
Traceback (most recent call last):
  File "/usr/local/bin/isrcsubmit.py", line 47, in <module>
    import discid
  File "/usr/local/lib/python3.6/dist-packages/discid/__init__.py", line 30, in <module>
    from discid.disc import read, put, Disc, DiscError, TOCError
  File "/usr/local/lib/python3.6/dist-packages/discid/disc.py", line 24, in <module>
    from discid.libdiscid import _LIB, FEATURES
  File "/usr/local/lib/python3.6/dist-packages/discid/libdiscid.py", line 108, in <module>
    _LIB = _open_library(_LIB_NAME)
  File "/usr/local/lib/python3.6/dist-packages/discid/libdiscid.py", line 99, in _open_library
    return ctypes.cdll.LoadLibrary(lib_name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libdiscid.so.0: cannot open shared object file: No such file or directory

I am not going to fix this now, as it would be a bit more complex.

Just for reference there is another alternative library, python-libdiscid which is using the same C library libdiscid. It is not using ctypes but is compiled against libdiscid directly. This fails on install:

# pip3 install --no-cache-dir python-libdiscid
Collecting python-libdiscid
  Downloading https://files.pythonhosted.org/packages/b8/be/88ffe177fef362fca0a12e67d400ce92cd96274c15cbe7d7d1674f8dec5c/python-libdiscid-1.0.tar.gz (64kB)
    100% |################################| 71kB 1.4MB/s 
Installing collected packages: python-libdiscid
  Running setup.py install for python-libdiscid ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_n7uygih/python-libdiscid/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-zeav4qaw-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/libdiscid
    copying libdiscid/discid.py -> build/lib.linux-x86_64-3.6/libdiscid
    copying libdiscid/exceptions.py -> build/lib.linux-x86_64-3.6/libdiscid
    copying libdiscid/__init__.py -> build/lib.linux-x86_64-3.6/libdiscid
    creating build/lib.linux-x86_64-3.6/libdiscid/tests
    copying libdiscid/tests/test_libdiscid.py -> build/lib.linux-x86_64-3.6/libdiscid/tests
    copying libdiscid/tests/test_compat_discid.py -> build/lib.linux-x86_64-3.6/libdiscid/tests
    copying libdiscid/tests/common.py -> build/lib.linux-x86_64-3.6/libdiscid/tests
    copying libdiscid/tests/__init__.py -> build/lib.linux-x86_64-3.6/libdiscid/tests
    creating build/lib.linux-x86_64-3.6/libdiscid/compat
    copying libdiscid/compat/discid.py -> build/lib.linux-x86_64-3.6/libdiscid/compat
    copying libdiscid/compat/__init__.py -> build/lib.linux-x86_64-3.6/libdiscid/compat
    running build_ext
    building 'libdiscid._discid' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/libdiscid
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c libdiscid/_discid.c -o build/temp.linux-x86_64-3.6/libdiscid/_discid.o
    In file included from libdiscid/_discid.c:445:0:
    libdiscid/discid-wrapper.h:25:10: fatal error: discid/discid.h: No such file or directory
     #include <discid/discid.h>
              ^~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_n7uygih/python-libdiscid/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-zeav4qaw-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-_n7uygih/python-libdiscid/

So it has the advantage to fail early, but it also needs the header files (which python-discid doesn't) and the error message you get might not be "straightforward".

@JonnyJD JonnyJD closed this as completed Feb 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants