library for Gridgen not found.

[bhattvihang]

I have recently installed PyRoms in my computer with python 3.7 from anaconda. I am running the software on MAC OS X Kernel Version 19.0.0. I created proms environment and all libraries (libcsa, libgridgen, libgd etc) are installed in /opt/anaconda3/envs/pyroms/lib/

import proms, import bath_smoother or import proms_toolbox does not create any error. however, when I try to generate grid I get following error.
In [10]: hgrd =pyroms.grid.Gridgen(lonp,latp,beta,(Mp+3,Lp+3),proj=map)

OSError Traceback (most recent call last)
in
----> 1 hgrd =pyroms.grid.Gridgen(lonp,latp,beta,(Mp+3,Lp+3),proj=map)

~/pyroms/pyroms/pyroms/hgrid.py in init(self, xbry, ybry, beta, shape, ul_idx, focus, proj, nnodes, precision, nppe, newton, thin, checksimplepoly, verbose)
1044
1045 #self._libgridgen = np.ctypeslib.load_library('libgridgen',file)
-> 1046 self._libgridgen = np.ctypeslib.load_library('libgridgen', pyroms.path[0])
1047
1048 # In MacOSX, use of c_void_p does not return proper structure.

~/opt/anaconda3/envs/pyroms/lib/python3.7/site-packages/numpy/ctypeslib.py in load_library(libname, loader_path)
155 raise
156 ## if no successful return in the libname_ext loop:
--> 157 raise OSError("no file with expected extension")
158
159 ctypes_load_library = deprecate(load_library, 'ctypes_load_library',

OSError: no file with expected extension

can anyone suggest what need to do in order to resolve the issue?

Thanks in advance.

Vihang Bhatt

[kshedstrom]

I get something similar on Linux:

python make_YELLOW_grd_v1.py
Traceback (most recent call last):
  File "make_YELLOW_grd_v1.py", line 30, in <module>
    hgrd = pyroms.grid.Gridgen(lonp, latp, beta, (Mm+3, Lm+3), proj=map)
  File "/import/home/kshedstrom/pyroms/pyroms/pyroms/hgrid.py", line 1046, in __init__
    self._libgridgen = np.ctypeslib.load_library('libgridgen', pyroms.__path__[0])
  File "/import/c1/AKWATERS/kate/.conda/envs/snowdrifts/lib/python3.7/site-packages/numpy/ctypeslib.py", line 157, in load_library
    raise OSError("no file with expected extension")
OSError: no file with expected extension
Exception ignored in: <function Gridgen.__del__ at 0x7f2788cacbf8>
Traceback (most recent call last):
  File "/import/home/kshedstrom/pyroms/pyroms/pyroms/hgrid.py", line 1163, in __del__
    self._libgridgen.gridnodes_destroy(self._gn)
AttributeError: 'Gridgen' object has no attribute '_libgridgen'

Hence the need for a test suite...

As for libgridgen, it took some fussing, including turning off strict channels to get it to install. Maybe having a warning about it not being available was better than failing to load it.

[hadfieldnz]

Hmm, interesting. I have not yet attempted to run that function so cannot verify whether this works on my Linux or Windows systems. I can merely make the following observations:

Line 1045 of pyroms/pyroms/pyroms/hgrid.py is

self._libgridgen = np.ctypeslib.load_library('libgridgen', pyroms.__path__[0])
Note the two underscores. The word "path" has been rendered in bold in your comment and the underscores omitted.

For me the path in question is this one
>>> print(pyroms.__path__[0]) /home/hadfield/Python/pyroms/pyroms/pyroms

It's looking for libgridgen. There is no such library in this directory on my system. The only files with a name containing "gridgen" on my system are these:

$ find /home/hadfield/Python/ -name '*gridgen*' /home/hadfield/Python/pyroms/pyroms/external/gridgen /home/hadfield/Python/pyroms/pyroms/external/gridgen/gridgen.c /home/hadfield/Python/pyroms/pyroms/external/gridgen/gridgen.h
So we need to build and install the gridgen library. There are instructions in
/home/hadfield/Python/pyroms/pyroms/external/gridgen/README

There's a gridgen by the same author on GitHub and a corresponding package on Conda-Forge. So please try installing that and if it works we'll add it to the list of dependencies.

Hi Kate. Your comment came through as I was typing. Test suite? Absolutely!

[kshedstrom]

Actually, Pavel Sakov recommended pygridgen to me instead. Good idea.

[hadfieldnz]

There are several such substitutions that could simplify and improve PyROMS. I think we should get a test suite going before we try them.

[bhattvihang]

Hi,

Thank you for all. The discussion has helped me to understand and identify the problem. I have solved it in following way as my efforts yesterday did not give any positive results. Apart from other usual steps suggested by developers, I followed following steps to reinstall the package.

Step 1: (out of disappointment)
removed all installations and environment used to install Pyroms including download git repository of Pyroms.

Step2
downloaded Pyroms again along with external dependencies csa, nn, gridgen, gridutils from Pavel Sakov git repo.

step3
installed all external dependencies in /usr/local

step 4:
changed the search path for libgridgen in hgrid as follows
try:
self._libgridgen = np.ctypeslib.load_library('libgridgen', pyroms.path[0])
except:
self._libgridgen = np.ctypeslib.load_library('libgridgen.so', pyroms.path[0])
and install Pyroms using setup.py

step 5:
As the installation path for Pyroms is '/opt/anaconda3/envs/pyroms/lib/python3.7/site-packages/pyroms' in my case, I manually created symbolic links here to libcsa, libgridgen, libgridutils installed in /usr/local/lib.

This solved the problem. I tested with YellowSea test that generated grid. Now I am looking into creating grid using GUI. any advise on how to invoke GUI to select domain will be useful.

Cheers and thank you again for looking into this issue.

Vihang Bhatt

[hadfieldnz]

I'm glad you've found a solution that works for you. I think that in the long term a Conda-installed gridgen will be simpler and more robust.

If gridgen is installed via Conda there's a libgridgen.so installed in the usual place, eg on my system:

$ find /home/hadfield/miniconda3/envs/python37/ -name '*gridgen*'
/home/hadfield/miniconda3/envs/python37/conda-meta/gridgen-1.51.5-h14c3975_1000.json
/home/hadfield/miniconda3/envs/python37/lib/libgridgen.so
/home/hadfield/miniconda3/envs/python37/lib/libgridgen.a
/home/hadfield/miniconda3/envs/python37/bin/gridgen
/home/hadfield/miniconda3/envs/python37/include/gridgen.h

This can be loaded if you supply the appropriate path:

>>> np.ctypeslib.load_library('libgridgen','/home/hadfield/miniconda3/envs/python37/lib')
<CDLL '/home/hadfield/miniconda3/envs/python37/lib/libgridgen.so', handle 55a991103970 at 0x7fc8fdd87208>

I will try check this solution out more thoroughly and, if successful, make some changes to the code and the documentation.

[balaji426]

I had similar issue with grid generation.
The problem got solved using hadfieldnz solution
Thank you very much.

Balaji