extractcode fails on Fedora 24

[akaihola]

On Fedora 24, I get this:

$ extractcode test/requests-2.11.1.tar.gz                        
Extracting archives...
  [------------------------------------]
Traceback (most recent call last):
  File "scancode-toolkit/bin/extractcode", line 9, in <module>
    load_entry_point('scancode-toolkit', 'console_scripts', 'extractcode')()
  File "scancode-toolkit/lib/python2.7/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "scancode-toolkit/src/scancode/utils.py", line 64, in main
    standalone_mode=standalone_mode, **extra)
  File "scancode-toolkit/lib/python2.7/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "scancode-toolkit/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "scancode-toolkit/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "scancode-toolkit/src/scancode/extract_cli.py", line 156, in extractcode
    for xev in extraction_events:
  File "scancode-toolkit/lib/python2.7/site-packages/click/_termui_impl.py", line 240, in next
    rv = next(self.iter)
  File "scancode-toolkit/src/scancode/api.py", line 43, in extract_archives
    from extractcode.extract import extract
  File "scancode-toolkit/src/extractcode/extract.py", line 37, in <module>
    from extractcode import archive
  File "scancode-toolkit/src/extractcode/archive.py", line 47, in <module>
    from extractcode import libarchive2
  File "scancode-toolkit/src/extractcode/libarchive2.py", line 91, in <module>
    libarchive = load_lib()
  File "scancode-toolkit/src/extractcode/libarchive2.py", line 79, in load_lib
    lib = ctypes.CDLL(libarchive)
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 357, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libbz2.so.1.0: cannot open shared object file: No such file or directory

It starts working correctly if I run this:

sudo ln -s /usr/lib64/libbz2.so.1 /usr/lib64/libbz2.so.1.0

[pombredanne]

@akaihola Thanks for the report. I will spin a Fedora 24 VM to reproduce the failure. The thing is that the prebuilt native libraries (here libarchive) are built on purpose on (very) old Linux distros for maximum compatibility. I think I should consider doing either a static build for the few deps of libarchive or vendor the deps built on the same machine. I reckon you were running on a 64 bits arch, right?

[pombredanne]

It looks like it may be a pervasive issue on RedHat-style distros: https://bugzilla.redhat.com/show_bug.cgi?id=461863 and https://www.google.com/search?q="libbz2.so.1.0"+in+Fedora ....

[pombredanne]

And more specifically on Fedora: http://forums.fedoraforum.org/showthread.php?t=258694 which seems to stem from the fact that the ScanCode binaries were produced on a debian or ubuntu distro.

[akaihola]

@pombredanne, yes, I'm running the 64-bit variant of Fedora 24.

[armijnhemel]

Would be nicer if instructions on how to replace the prebuilt binaries were added :-)

[pombredanne]

@armijnhemel agreed. I am working on something there.