auditwheel error on bigartm wheel

[ofrei]

pr #830 implement a feature that build a python wheel. I've tried to run auditwheel, but it fails:

RuntimeError: Invalid binary wheel, found shared library "libartm.so" in purelib folder.
The wheel has to be platlib compliant in order to be repaired by auditwheel.

Unpacking the wheel shows that it has the following structure:

bigartm-0.8.3.data
  purelib
    artm
      *.py (bunch of .py files which constitutes python API of bigartm)
      libartm.so 
bigartm-0.8.3.dist-info

I'm surprised to see a purelib here. I would expect that our artm folder goes into the root, and we don't have bigartm-0.8.3.data folder. I'm not quite sure, but I've looked into numpy wheel, and it seems to be different -- they barely use .data folder, and there is numpy folder under the root, which also contains .so files.

@vmarkovtsev do you know how to fix this? I'm not quire sure how BinaryDistribution(Distribution) and AddLibraryBuild(build_py) work. Perhaps the issue is how does setup.py bdist_wheel detect that library is platlib and not a purelib? They say it is automatic, but how it works?

bdist_wheel will detect that the code is not pure Python, and build a wheel that’s named such that it’s only usable on the platform that it was built on

[vmarkovtsev]

I will look into this, weird, this setup worked for one of our projects.

[ofrei]

Thanks! I have a good progress with building bigartm on manylinux docker image (and it seems to work - I've tested the resulting wheel in ubuntu:14.04 image). So as long as auditwheel works I think I'll be able to push those wheels on PyPi. The only trouble is, how to build wheels for mac? Do I have to buy a mac?.. Anyway, let's first sort out auditwheel first :)

[aborg-dev]

You can try to use Travis OSX build environment: https://docs.travis-ci.com/user/reference/osx/

[JeanPaulShapo]

I think there is problem in implementaion of install command in standard distutils library in finalize_options:

# Pick the actual directory to install all modules to: either
# install_purelib or install_platlib, depending on whether this
# module distribution is pure or not.  Of course, if the user
# already specified install_lib, use their selection.
if self.install_lib is None:
    if self.distribution.ext_modules: # has extensions: non-pure
        self.install_lib = self.install_platlib
    else:
        self.install_lib = self.install_purelib

In order to install into platform-dependent directory, you must have at least one extension, but they don't check for has_ext_modules function! That's why there is some weird behaviour.

One of the possible solutions is to override default behaviour in custom subclass of install command:

class InstallPlatlib(install):
    def finalize_options(self):
        install.finalize_options(self)
        if self.distribution.has_ext_modules():
            self.install_lib = self.install_platlib

[ofrei]

@JeanPaulShapo Thanks! I'll give it a try...

[zurk]

@ofrei, I discovered the same problem for https://github.com/src-d/minhashcuda project
I was able to solve it with older auditwheel version.
try to use 1.4 or 1.5. If you decide to use 1.5 you should also update patchelf Debian package to 0.9 version.

Good luck!

[ofrei]

@zurk Thanks! I'll give it a try

[bt2901]

Found easy workaround: pypa/auditwheel#28 (comment)

You should create setup.cfg in /bigartm/python with the following text:

[install]
install_lib=

With cautious optimism, here's a provisional wheel file: https://www.dropbox.com/s/ju3s7p1sac9kpem/bigartm-0.9.0-cp37-cp37m-manylinux1_x86_64.whl?dl=0