ImportError: libGL.so.1: cannot open shared object file: No such file or directory

[rsignell-usgs]

on Ubuntu 16.04.3 LTS:

$ conda create -n pygridgen python=3 pygridgen
$ source activate pygridgen
$ python
Python 3.6.4 | packaged by conda-forge | (default, Dec 23 2017, 16:31:06)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygridgen
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/pygridgen/__init__.py", line 17, in <module>
    from .boundary_interactor import *
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/pygridgen/boundary_interactor.py", line 12, in <module>
    import matplotlib.pyplot as plt
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/pyplot.py", line 116, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/__init__.py", line 60, in pylab_setup
    [backend_name], 0)
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/backend_qt5agg.py", line16, in <module>
    from .backend_qt5 import (
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/backend_qt5.py", line 18, in <module>
    import matplotlib.backends.qt_editor.figureoptions as figureoptions
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/qt_editor/figureoptions.py", line 20, in <module>
    import matplotlib.backends.qt_editor.formlayout as formlayout
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/qt_editor/formlayout.py", line 56, in <module>
    from matplotlib.backends.qt_compat import QtGui, QtWidgets, QtCore
  File "/home/jovyan/my-conda-envs/pygridgen/lib/python3.6/site-packages/matplotlib/backends/qt_compat.py", line 137,in <module>
    from PyQt5 import QtCore, QtGui, QtWidgets
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

$ conda list
# packages in environment at /home/jovyan/my-conda-envs/pygridgen:
#
backports                 1.0                      py36_1    conda-forge
backports.functools_lru_cache 1.5                      py36_0    conda-forge
ca-certificates           2018.1.18                     0    conda-forge
certifi                   2018.1.18                py36_0    conda-forge
csa                       1.22.0                        2    conda-forge
cycler                    0.10.0                   py36_0    conda-forge
dbus                      1.10.22                       0    conda-forge
expat                     2.2.5                         0    conda-forge
fontconfig                2.12.6                        0    conda-forge
freetype                  2.8.1                         0    conda-forge
gettext                   0.19.8.1                      0    conda-forge
glib                      2.55.0                        0    conda-forge
gridgen                   1.51.2                        2    conda-forge
gridutils                 1.0                           2    conda-forge
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         1    conda-forge
icu                       58.2                          0    conda-forge
intel-openmp              2018.0.0             hc7b2577_8    defaults
jpeg                      9b                            2    conda-forge
libffi                    3.2.1                         3    conda-forge
libgcc-ng                 7.2.0                h7cc24e2_2    defaults
libgfortran-ng            7.2.0                h9f7466a_2    defaults
libiconv                  1.15                          0    conda-forge
libpng                    1.6.34                        0    conda-forge
libxcb                    1.12                          1    conda-forge
libxml2                   2.9.7                         0    conda-forge
matplotlib                2.1.2                    py36_0    conda-forge
mkl                       2018.0.1             h19d6760_4    defaults
ncurses                   5.9                          10    conda-forge
nn                        1.86.0                        2    conda-forge
numpy                     1.14.0           py36h3dfced4_1    defaults
openssl                   1.0.2n                        0    conda-forge
pandas                    0.22.0                   py36_0    conda-forge
pcre                      8.39                          0    conda-forge
pip                       9.0.1                    py36_1    conda-forge
pygridgen                 0.1.1                    py36_1    conda-forge
pyparsing                 2.2.0                    py36_0    conda-forge
pyproj                    1.9.5.1                  py36_0    conda-forge
pyqt                      5.6.0                    py36_4    conda-forge
python                    3.6.4                         0    conda-forge
python-dateutil           2.6.1                    py36_0    conda-forge
pytz                      2018.3                     py_0    conda-forge
qt                        5.6.2                         7    conda-forge
readline                  7.0                           0    conda-forge
setuptools                38.5.1                   py36_0    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.11.0                   py36_1    conda-forge
sqlite                    3.20.1                        2    conda-forge
tk                        8.6.7                         0    conda-forge
tornado                   4.5.3                    py36_0    conda-forge
wheel                     0.30.0                   py36_2    conda-forge
xorg-libxau               1.0.8                         3    conda-forge
xorg-libxdmcp             1.1.2                         3    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.11                        0    conda-forge

[rsignell-usgs]

anaconda/docker-images#49 (comment)

[rsignell-usgs]

I was able to workaround this issue on my system by doing:

sudo apt update
sudo apt install libgl1-mesa-glx

but it would be nice to get these requirements in the conda package.

[ocefpaf]

@rsignell-usgs we, conda-forge, considered creating a qt recipe that did not depend on system libs. However, continuum's policy for qt does not do that for many reasons*. There are valid points on both sides and ideally we should support both so people using docker images like yours do not need to workaround it like that.

The problem is that, when conda-forge added the extra dependencies our qt is never selected by conda's solver. Conda's solver tries to solve the "easiest" path, even ignores the channel preference in order to accomplish that, so only defaults qt was installed, rendering the conda-forge alternative qt useless. (I experimented with that for a while and reverted the changes b/c of that.)

TL;DR closing this b/c there is not much we can do for now.

[rsignell-usgs]

So I should raise this issue with anaconda to get the defaults package fixed?

[rsignell-usgs]

Looks like they've been trying!
https://anaconda.org/search?q=mesa-libGL

[ocefpaf]

So I should raise this issue with anaconda to get the defaults package fixed?

It is not really a "fix," more of a policy question. Ship lower level system libs or not. @msarahan and @mingwandroid can comment more on that.

https://anaconda.org/search?q=mesa-libGL

Those are new, together and goes together with the new compilers, but I am out of the loop to comment how they'll be used. I do know that @mingwandroid is on top of things and he is trying to accommodate everyone's needs in the best solution possible.

PS: however, the fact that the conda solver hinders an alternative qt due to the extra deps like I mentioned above, is something that could be fixed... But that is not the issue here.

[mingwandroid]

We do not wish to ship such low level libraries as they interface with end-users' hardware (but we could do better with advising people which system level package manager packages they need to install).

mesa-libGL is software only and not appropriate for people who have spent money on a nVidia GTX 1080 or an AMD Vega GPU. You would not be able to do anything graphically intensive with a Qt built in this way (see the upcoming Qt3d for example or even any old QtOpenGL stuff - and some people may want to use shader-based GPGPU to do some computation - though there are far better ways!)

Looks like they've been trying!

No, we haven't. Those packages are direct repackages of CentOS6's mesa-GL. We do this so that our compilers find these libraries and do not then find (for example) newer system GL libraries that causes the software we build to not work on CentOS6. We call these packages 'Core Dependency Tree' packages and they are never installed at runtime (only ever at build-time).

@kalefranz, any comment on the solver issue? From what I know, I really dislike this minimize number of packages (or is it minimize change of number of packages?) heuristic as it naturally prefers less well split up package sets and static libraries over shared ones.

[ocefpaf]

@mingwandroid thanks for the explanation and sorry for not knowing it by heart now (you already told me that a few times 😬)

Actually, I will write that as docs/wiki so we can direct people to it. This questions does comes every now and then.

[rsignell-usgs]

@ocefpaf, so what is the preferred solution for someone like me, who is trying to create an environment for JupyterHub running on Google Cloud?

[ocefpaf]

So what is the preferred solution for someone like me, who is trying to create an environment for JupyterHub running on Google Cloud?

Get the libs with apt-get like you are already doing. There is no all-conda solution :-(

[mingwandroid]

Put the system package install commands into in your Dockerfile or whatever?

[rsignell-usgs]

Okay, thanks. I will request that mod in the Dockerfile.

[mingwandroid]

My attempts to reproduce this failed. I am able to create an env with conda-forge's qt using:

conda create -n pygridgen python=3 pygridgen

.. so long as I have conda-forge above defaults in my ~/.condarc and also if I have no ~/.condarc and instead use:

conda create -n pygridgen python=3 pygridgen -c conda-forge

[ocefpaf]

My attempts to reproduce this failed. I am able to create an env with conda-forge's qt using

Our qt is the same as defaults now @mingwandroid, so we will have the same issue on a docker image. I removed the qts where I experimented with adding the extra lower level libs b/c of the conda solver issue.

[ghost]

I use sudo apt install libgl1-mesa-glx
then conda kernal dies.

~/anaconda3/envs/venv36/lib/python3.6/site-packages/matplotlib/backends/backend_qt5.py in _create_qApp()
    123                 display = os.environ.get('DISPLAY')
    124                 if display is None or not re.search(r':\d', display):
--> 125                     raise RuntimeError('Invalid DISPLAY variable')
    126 
    127             qApp = QtWidgets.QApplication([b"matplotlib"])

RuntimeError: Invalid DISPLAY variable

[phobson]

@tyan4g try setting your matplotlib backend to 'agg'

https://matplotlib.org/faq/usage_faq.html#what-is-a-backend

[megies]

For future reference, the corresponding paragraph in the conda-forge docs can be found here: https://conda-forge.org/docs/maintainer/knowledge_base.html#libgl

[relh]

If you need libGL for compiling things you should probably use this "devel" one to avoid having to also install a 100 other packages manually when make fails on them: https://anaconda.org/conda-forge/mesa-libgl-devel-cos7-x86_64