Issue installing cfgrib with eccodes 2.19.0

[chris-haste]

Hi,
I am running Ubuntu 20.04.1 LTS
I have built eccodes version 2.19.0 following the instructions here
https://confluence.ecmwf.int/display/ECC/ecCodes+installation

and installed it to /usr/local
the tests all pass.

I have installed cfgrib but, when I run

python -m cfgrib selfcheck

I get the following error:

Traceback (most recent call last):
  File "/home/chris/grib/venv/lib/python3.8/site-packages/cfgrib/bindings.py", line 56, in <module>
    lib = ffi.dlopen(libname)
  File "/home/chris/grib/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/home/chris/grib/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/home/chris/grib/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
    raise OSError(msg)
OSError: ctypes.util.find_library() did not manage to locate a library called 'libeccodes'

the libeccodes.so file is located here:

(venv) chris@DESKTOP-SDMIGPM:~/grib$ ls /usr/local/lib/
cmake  libopenjp2.a  libopenjp2.so.2.3.1  openjpeg-2.3  python3.8
libeccodes.so  libopenjp2.so  libopenjp2.so.7 pkgconfig

Any idea why cfgrib cant find it?

[iainrussell]

This might not be relevant, but I'd expect eccodes 2.19.0 to install its library into /usr/local/lib64 by default, not lib. It might be worth checking whether the library in /usr/local/lib really is 2.19.0 and not the remnants of a previous installation. We've seen issues when it's installed into a lib64 directory.
See #167

[chris-haste]

Hi and thanks for the quick reply.
I had previously forced the install to /usr/local but I have just started a new instance and this time allowed the eccodes build to use its default.
The default build on eccodes 2.19.0 does install to /usr/local if no CMAKE_INSTALL_PREFIX option is specified.

So, next I set up a venv and install numpy, pyproj. Followed by

pip3 install eccodes

now when I run

python3 -m eccodes selfcheck

I get a different error.

Traceback (most recent call last):
  File "/home/chris/venv/lib/python3.8/site-packages/gribapi/bindings.py", line 29, in <module>
    from ._bindings import ffi, lib
ModuleNotFoundError: No module named 'gribapi._bindings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chris/venv/lib/python3.8/site-packages/gribapi/bindings.py", line 53, in <module>
    lib = ffi.dlopen(libname)
  File "/home/chris/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/home/chris/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/home/chris/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
    raise OSError(msg)
OSError: cannot load library '/usr/local/lib64/libeccodes.so': /usr/local/lib64/libeccodes.so: cannot open shared object file: No such file or directory.  Additionally, ctypes.util.find_library() did not manage to locate a library called '/usr/local/lib64/libeccodes.so'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/chris/venv/lib/python3.8/site-packages/eccodes/__init__.py", line 15, in <module>
    from .eccodes import *
  File "/home/chris/venv/lib/python3.8/site-packages/eccodes/eccodes.py", line 12, in <module>
    from gribapi import __version__
  File "/home/chris/venv/lib/python3.8/site-packages/gribapi/__init__.py", line 13, in <module>
    from .gribapi import *  # noqa
  File "/home/chris/venv/lib/python3.8/site-packages/gribapi/gribapi.py", line 32, in <module>
    from .bindings import ENC, ffi, lib
  File "/home/chris/venv/lib/python3.8/site-packages/gribapi/bindings.py", line 60, in <module>
    raise RuntimeError(f"ecCodes library not found using {LIBNAMES}")
RuntimeError: ecCodes library not found using ['/usr/local/lib64/libeccodes.so', '/usr/local/lib/libeccodes.so', 'eccodes', 'libeccodes.so', 'libeccodes']

So an error about the bindings and now an indication that it is looking in lib64 for libeccodes.so

My system does not have a /usr/local/lib64 directory. I can confirm that the contents of my /usr/local/lib are:

(venv) chris@DESKTOP-F6MIGPM:~$ ls /usr/local/lib/
cmake          libopenjp2.a   libopenjp2.so.2.3.1  openjpeg-2.3  python3.8
libeccodes.so  libopenjp2.so  libopenjp2.so.7      pkgconfig

Also, and maybe not related at all, but I am not entirely sure what environment variables I should use/set. Currently I have these two:

(venv) chris@DESKTOP-F6MIGPM:~$ export
declare -x ECCODES_DEFINITION_PATH="/usr/local/share/eccodes/definitions"
declare -x ECCODES_DIR="/usr/local"

[alexamici]

@chris-haste the ECCODES_DIR is only supported in master that is not released yet. The upcoming version 0.9.8.5 will support overriding the library location with the environment variables.