grass.session error after merge of #14

[neteler]

After the recent merge of #14 the GRASS GIS docker images no longer build:

https://hub.docker.com/repository/registry-1.docker.io/neteler/docker-alpine-grass-gis/builds/f2496f54-56f6-4793-93f5-af27dffbd452

Step 33/50 : RUN pip3 install --upgrade pip six grass-session
---> Running in aa2ea2f0afb6
Requirement already up-to-date: pip in /usr/lib/python3.8/site-packages (20.0.2)
Collecting six
Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting grass-session
Downloading grass-session-0.4.tar.gz (9.0 kB)
�[91m ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x9mi7d74/grass-session/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x9mi7d74/grass-session/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-x9mi7d74/grass-session/pip-egg-info
cwd: /tmp/pip-install-x9mi7d74/grass-session/
Complete output (12 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-x9mi7d74/grass-session/setup.py", line 11, in <module>
import grass_session # isort:skip
File "/tmp/pip-install-x9mi7d74/grass-session/grass_session/__init__.py", line 4, in <module>
from grass_session.session import (
File "/tmp/pip-install-x9mi7d74/grass-session/grass_session/session.py", line 476, in <module>
load_libs(GISBASE)
File "/tmp/pip-install-x9mi7d74/grass-session/grass_session/session.py", line 336, in load_libs
raise RuntimeError("No GRASS GIS libraries found in {}.".format(ld_path))
RuntimeError: No GRASS GIS libraries found in /usr/local/grass79/lib.
Loading libraries from /usr/local/grass79/lib

Do we have to update anything here?

https://github.com/OSGeo/grass/blob/master/docker/alpine/Dockerfile_alpine#L195

Thanks...

[neteler]

Maybe @zarch or @ninsbl have an idea?

[ninsbl]

Hmmm… On Ubuntu 18.04 With GRASS GIS 7.8 I get a similar error Message:

pip install --user grass-session
Collecting grass-session
  Using cached https://files.pythonhosted.org/packages/16/7a/0f399629b0a5243dc2e64f3bb145188de5133b06bd61f308966615290b82/grass-session-0.4.tar.gz
Building wheels for collected packages: grass-session
  Running setup.py bdist_wheel for grass-session ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-n_rdKy/grass-session/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp_g8cACpip-wheel- --python-tag cp27:
  Loading libraries from /usr/lib/grass78/lib
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for grass-session
  Running setup.py clean for grass-session
Failed to build grass-session
Installing collected packages: grass-session
  Running setup.py install for grass-session ... done
Successfully installed grass-session-0.4

On that machine, the following command installs grass-session without issues:
python3 -m pip install --user grass-session
Python Version is 3.6.9

Also, on another Ubuntu box, grass-session 0.4 installs without issue both with python2 and python3 (3.6.8).

[ninsbl]

On Fedora I have no issues installing grass-session either...

The function that throws the error uses ctypes. Maybe there is the culprit?

[mmacata]

In above alpine GRASS GIS Dockerimage, I tested python3 -m pip install grass-session without success. Same error as above.

[zarch]

@neteler thank you to open this issue, I was not aware of this problem, I'm on fedora right now and the installation it seems to be fine.

The issue is that grass is providing the wrong GISBASE, the command:

# grass --config path
/usr/local/grass79

Return this directory that do not exist.

A possible solution to fix this could be to move the installation of grass-session after the proper link to the GISBASE directory are created, see this commit.

However now the installation process is failing because call the ctypes function ctypes.CDLL(lib, mode=1) in the load_libs function raise an OSError:

Step 37/52 : RUN pip3 install --upgrade pip six grass-session
 ---> Running in 073ae793bf9e
Requirement already up-to-date: pip in /usr/lib/python3.8/site-packages (20.0.2)
Collecting six
  Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting grass-session
  Downloading grass-session-0.4.tar.gz (9.0 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ia21xtta/grass-session/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ia21xtta/grass-session/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-ia21xtta/grass-session/pip-egg-info
         cwd: /tmp/pip-install-ia21xtta/grass-session/
    Complete output (12 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-ia21xtta/grass-session/setup.py", line 11, in <module>
        import grass_session  # isort:skip
      File "/tmp/pip-install-ia21xtta/grass-session/grass_session/__init__.py", line 4, in <module>
        from grass_session.session import (
      File "/tmp/pip-install-ia21xtta/grass-session/grass_session/session.py", line 476, in <module>
        load_libs(GISBASE)
      File "/tmp/pip-install-ia21xtta/grass-session/grass_session/session.py", line 355, in load_libs
        raise RuntimeError(
    RuntimeError: Cannot load all the following GRASS GIS libraries from ['/usr/local/grass79/lib/libgrass_arraystats.7.9.so', '/usr/local/grass79/lib/libgrass_arraystats.so', '/usr/local/grass79/lib/libgrass_bitmap.7.9.so', '/usr/local/grass79/lib/libgrass_bitmap.so', '/usr/local/grass79/lib/libgrass_btree2.7.9.so', '/usr/local/grass79/lib/libgrass_btree2.so', '/usr/local/grass79/lib/libgrass_cairodriver.7.9.so', '/usr/local/grass79/lib/libgrass_cairodriver.so', '/usr/local/grass79/lib/libgrass_calc.7.9.so', '/usr/local/grass79/lib/libgrass_calc.so', '/usr/local/grass79/lib/libgrass_cluster.7.9.so', '/usr/local/grass79/lib/libgrass_cluster.so', '/usr/local/grass79/lib/libgrass_dbmibase.7.9.so', '/usr/local/grass79/lib/libgrass_dbmibase.so', '/usr/local/grass79/lib/libgrass_dbmiclient.7.9.so', '/usr/local/grass79/lib/libgrass_dbmiclient.so', '/usr/local/grass79/lib/libgrass_dbmidriver.7.9.so', '/usr/local/grass79/lib/libgrass_dbmidriver.so', '/usr/local/grass79/lib/libgrass_dbstubs.7.9.so', '/usr/local/grass79/lib/libgrass_dbstubs.so', '/usr/local/grass79/lib/libgrass_dig2.7.9.so', '/usr/local/grass79/lib/libgrass_dig2.so', '/usr/local/grass79/lib/libgrass_display.7.9.so', '/usr/local/grass79/lib/libgrass_display.so', '/usr/local/grass79/lib/libgrass_driver.7.9.so', '/usr/local/grass79/lib/libgrass_driver.so', '/usr/local/grass79/lib/libgrass_dspf.7.9.so', '/usr/local/grass79/lib/libgrass_dspf.so', '/usr/local/grass79/lib/libgrass_g3d.7.9.so', '/usr/local/grass79/lib/libgrass_g3d.so', '/usr/local/grass79/lib/libgrass_gis.7.9.so', '/usr/local/grass79/lib/libgrass_gis.so', '/usr/local/grass79/lib/libgrass_gmath.7.9.so', '/usr/local/grass79/lib/libgrass_gmath.so', '/usr/local/grass79/lib/libgrass_gpde.7.9.so', '/usr/local/grass79/lib/libgrass_gpde.so', '/usr/local/grass79/lib/libgrass_gproj.7.9.so', '/usr/local/grass79/lib/libgrass_gproj.so', '/usr/local/grass79/lib/libgrass_htmldriver.7.9.so', '/usr/local/grass79/lib/libgrass_htmldriver.so', '/usr/local/grass79/lib/libgrass_imagery.7.9.so', '/usr/local/grass79/lib/libgrass_imagery.so', '/usr/local/grass79/lib/libgrass_interpfl.7.9.so', '/usr/local/grass79/lib/libgrass_interpfl.so', '/usr/local/grass79/lib/libgrass_iortho.7.9.so', '/usr/local/grass79/lib/libgrass_iortho.so', '/usr/local/grass79/lib/libgrass_lidar.7.9.so', '/usr/local/grass79/lib/libgrass_lidar.so', '/usr/local/grass79/lib/libgrass_lrs.7.9.so', '/usr/local/grass79/lib/libgrass_lrs.so', '/usr/local/grass79/lib/libgrass_manage.7.9.so', '/usr/local/grass79/lib/libgrass_manage.so', '/usr/local/grass79/lib/libgrass_neta.7.9.so', '/usr/local/grass79/lib/libgrass_neta.so', '/usr/local/grass79/lib/libgrass_pngdriver.7.9.so', '/usr/local/grass79/lib/libgrass_pngdriver.so', '/usr/local/grass79/lib/libgrass_psdriver.7.9.so', '/usr/local/grass79/lib/libgrass_psdriver.so', '/usr/local/grass79/lib/libgrass_raster.7.9.so', '/usr/local/grass79/lib/libgrass_raster.so', '/usr/local/grass79/lib/libgrass_rli.7.9.so', '/usr/local/grass79/lib/libgrass_rli.so', '/usr/local/grass79/lib/libgrass_rowio.7.9.so', '/usr/local/grass79/lib/libgrass_rowio.so', '/usr/local/grass79/lib/libgrass_rtree.7.9.so', '/usr/local/grass79/lib/libgrass_rtree.so', '/usr/local/grass79/lib/libgrass_segment.7.9.so', '/usr/local/grass79/lib/libgrass_segment.so', '/usr/local/grass79/lib/libgrass_sim.7.9.so', '/usr/local/grass79/lib/libgrass_sim.so', '/usr/local/grass79/lib/libgrass_stats.7.9.so', '/usr/local/grass79/lib/libgrass_stats.so', '/usr/local/grass79/lib/libgrass_symb.7.9.so', '/usr/local/grass79/lib/libgrass_symb.so', '/usr/local/grass79/lib/libgrass_temporal.7.9.so', '/usr/local/grass79/lib/libgrass_temporal.so', '/usr/local/grass79/lib/libgrass_vector.7.9.so', '/usr/local/grass79/lib/libgrass_vector.so', '/usr/local/grass79/lib/libgrass_vedit.7.9.so', '/usr/local/grass79/lib/libgrass_vedit.so']!
    Loading libraries from /usr/local/grass79/lib
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The command '/bin/sh -c pip3 install --upgrade pip six grass-session' returned a non-zero code: 1

Do you have an idea on why python fail loading the grass libs with ctypes?
The error is:

OSError: Error loading shared library libgrass_gis.7.9.so: No such file or directory (needed by /usr/local/grass/lib/libgrass_arraystats.7.9.so)

For every lib listed above.

[neteler]

The docker builds are all failing :(

Any chance to revert this PR soon?

[zarch]

@neteler ok, I released a new version now the load_libs function is not call by default.
In this way together with @ninsbl we can further investigate the issue.
Please let me know if the new version fix your issues (from my tests it should) :-)

[neteler]

Excellent, thanks a lot: I have updated all Dockerfile in master (OSGeo/grass@745ee7e) and releasebranch78 (OSGeo/grass@4f2f21b) accordingly.
Now waiting for docker hub to finish with all builds...

[neteler]

Success! Thanks again @zarch