3.3.0.10 breaks on ubuntu server without X11

[owend]

We're running an opencv python application in docker (with GPU support via nvidia-docker but no GUI). The most recent release has added a dependency on Qt, but it is also linking against X11, which is not typically installed in such environments. Would it be possible to provide a headless version, or include X11 and other libs in the build (which would make the resulting package a fair bit larger I'd imagine).

In version opencv_python-3.3.0.9-cp35-cp35m-manylinux1_x86_64.whl (8.8MB), ldd shows the following.

root@3b68495a9aa8:/usr/local/lib/python3.5/dist-packages/cv2# ldd cv2.cpython-35m-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007ffc9d7aa000)
libz-a147dcb0.so.1.2.3 => /usr/local/lib/python3.5/dist-packages/cv2/./.libs/libz-a147dcb0.so.1.2.3 (0x00007fe2c6f04000)
libgthread-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007fe2c6cfb000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fe2c69ea000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe2c6668000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe2c6463000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe2c6246000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe2c603e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe2c5d34000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe2c5b1e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe2c5754000)
/lib64/ld-linux-x86-64.so.2 (0x00005569d85ea000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe2c54e3000)

In version opencv_python-3.3.0.10-cp35-cp35m-manylinux1_x86_64.whl (15.4MB) ldd shows the following:

root@a867b910c313:/usr/local/lib/python3.5/dist-packages/cv2# ldd cv2.cpython-35m-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007fffbed3e000)
libz-a147dcb0.so.1.2.3 => /usr/local/lib/python3.5/dist-packages/cv2/./.libs/libz-a147dcb0.so.1.2.3 (0x00007f2862370000)
libQtGui-6d0f14dd.so.4.8.7 => /usr/local/lib/python3.5/dist-packages/cv2/./.libs/libQtGui-6d0f14dd.so.4.8.7 (0x00007f28614de000)
libQtTest-1183da5d.so.4.8.7 => /usr/local/lib/python3.5/dist-packages/cv2/./.libs/libQtTest-1183da5d.so.4.8.7 (0x00007f28612b1000)
libQtCore-ba1dc80c.so.4.8.7 => /usr/local/lib/python3.5/dist-packages/cv2/./.libs/libQtCore-ba1dc80c.so.4.8.7 (0x00007f2860da5000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f2860a1c000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2860818000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f28605fb000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f28603f2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f28600e9000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f285fed3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f285fb08000)
/lib64/ld-linux-x86-64.so.2 (0x000055aedf75a000)
libgthread-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007f285f906000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f285f5f5000)
libSM.so.6 => not found
libICE.so.6 => not found
libXrender.so.1 => not found
libXext.so.6 => not found
libX11.so.6 => not found
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f285f382000)

Running the application results in the following error:

File "/usr/local/lib/python3.5/dist-packages/cv2/init.py", line 9, in
from .cv2 import *
ImportError: libSM.so.6: cannot open shared object file: No such file or directory

[skvark]

Can you install libSM / other deps to the Docker image? The X11 dependencies come from Qt and they cannot be avoided.

Bundling libSM and other X11 libs into the wheels is not possible, because auditwheel excludes them automatically:

• https://github.com/pypa/auditwheel/blob/master/auditwheel/policy/policy.json
• https://www.python.org/dev/peps/pep-0513/#the-manylinux1-policy

Headless version should be possible to implement by patching the binary with patchelf (by removing problematic DT_NEEDED entries) and creating an alternate distribution. I'll try that when I have more time to focus on this project.

In the meantime, you can downgrade to 3.3.0.9 which is exactly the same build but without Qt.

[owend]

Thanks for the info. For now, a downgrade solves the immediate issue. If necessary, we can add X11 libs, but I'd prefer not to since we aren't using that functionality.

[rmekdma]

Same problem to me. I use python 2.7 and ubuntu 14.04 in the docker image. After I installed libsm6 another error came out.

  File "/usr/local/lib/python2.7/dist-packages/cv2/__init__.py", line 9, in <module>
    from .cv2 import *
ImportError: libXrender.so.1: cannot open shared object file: No such file or directory

[cipri-tom]

@rmekdma if you don't need graphical Qt stuff, you can install the very very slightly older build, like this: pip install 'opencv-contrib-python==3.3.0.9' (after removing the current one, of course: pip uninstall opencv-contrib-python)

[berak]

@skvark , installing libSM is not an option in environments like heroku, or google cloud engine (you're not root, and thus can't install anything or even build from src)

having qt support is nice on the desktop, but would it be possible to have a "headless" version ?

[skvark]

Yes, I will make a new headless entry after other blocking issues are fixed.

[skvark]

There are now two new headless distributions: https://github.com/skvark/opencv-python/releases/tag/14

Please note that it takes still a few hours before builds finish and packages are available for install.