[Bug] warning and error messages by g.extension while installing (nested) addons with helper libs #1499

hellik · 2021-04-03T09:06:22Z

Describe the bug

installing (nested) addons with helper libs like (e.g. i.sentinel, i.fusion.hpf, r.green) via g.extensions produces a lot of warning and error messages

To Reproduce
Steps to reproduce the behavior:

g.extension extension=i.sentinel                                                
Downloading precompiled GRASS Addons <i.sentinel>...
Fetching <i.sentinel> from <http://wingrass.fsv.cvut.cz/grass78/x86_64/addons/grass-7.8.5/i.sentinel.zip> (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'i.sentinel.download'.
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\\Users\\youruser\\AppData\\Roaming\\GRASS7\\addons\\docs\\html\\i.sentinel.download.html'

g.extension extension=i.fusion.hpf                                              
Downloading precompiled GRASS Addons <i.fusion.hpf>...
Fetching <i.fusion.hpf> from <http://wingrass.fsv.cvut.cz/grass79/x86_64/addons/grass-7.9.dev/i.fusion.hpf.zip> (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'constants'.
WARNING: No metadata available for module 'high_pass_filter'.
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\\Users\\youruser\\AppData\\Roaming\\GRASS7\\addons\\docs\\html\\constants.html'

g.extension extension=r.green                                                   
Downloading precompiled GRASS Addons <r.green>...
Fetching <r.green> from <http://wingrass.fsv.cvut.cz/grass79/x86_64/addons/grass-7.9.dev/r.green.zip> (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'plant'.
WARNING: No metadata available for module 'utils'.
WARNING: No metadata available for module 'r.green.hydro.technical'.
WARNING: No metadata available for module 'r.green.hydro.structure'.
WARNING: No metadata available for module 'harvesting'.
WARNING: No metadata available for module 'ashrae'.
WARNING: No metadata available for module 'financial'.
WARNING: No metadata available for module 'basin'.
WARNING: No metadata available for module 'r.green.hydro.planning'.
WARNING: No metadata available for module 'r.green.hydro.delplants'.
WARNING: No metadata available for module 'optimal'.
WARNING: No metadata available for module '__init__'.
WARNING: No metadata available for module 'checkparameter'.
WARNING: No metadata available for module 'r.green.hydro.theoretical'.
WARNING: No metadata available for module 'gpot'.
WARNING: No metadata available for module 'r.green.hydro.optimal'.
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\\Users\\youruser\\AppData\\Roaming\\GRASS7\\addons\\docs\\html\\plant.html'

Expected behavior

no warning and error messages for python libs of the addons (e.g. plant, utils in r.green or constans in i.fusion.hpf)

System description (please complete the following information):

Operating System: [e.g. Windows, Linux, ..., incl. version]
GRASS Version: 7.8.5
GRASS Version: 7.9.dev

The text was updated successfully, but these errors were encountered:

hellik · 2021-04-03T09:07:02Z

not sure if it's the same in linux?

veroandreo · 2021-04-03T15:12:46Z

In my Fedora 33, I get no errors nor warnings for i.fusion.hpf, but I do get the WARNING: No metadata available for module 'bla' when installing nested add-ons

stevenpawley · 2021-04-16T22:33:44Z

This is definitely a Windows issue, but a real problem if you are working on Windows because potentially none of the addons with nested modules can be installed.

I just tried installed two addons, r.learn.ml2, and v.class.ml on a completely fresh Windows VM. Both resulted in errors like below. It appears that g.extension is looking for HTML docs to be associated with every python module within the addon, e.g. it is looking for a 'raster.html' within r.learn.ml2 but 'raster' is only a python module that is imported by the addon.

g.extension extension=r.learn.ml2
Downloading precompiled GRASS Addons <r.learn.ml2>...
Fetching <r.learn.ml2> from http://wingrass.fsv.cvut.cz/grass79/x86_64/addons/grass-7.9.dev/r.learn.ml2.zip (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'raster': Unable to fetch interface description for command ''.

Details:
WARNING: No metadata available for module 'indexing': Unable to fetch interface description for command ''.

Details:
WARNING: No metadata available for module 'r.learn.predict': Unable to fetch interface description for command '<r.learn.predict>'.

Details:
WARNING: No metadata available for module 'r.learn.train': Unable to fetch interface description for command '<r.learn.train>'.

Details:
WARNING: No metadata available for module 'transformers': Unable to fetch interface description for command ''.

Details:
WARNING: No metadata available for module 'stats': Unable to fetch interface description for command ''.

Details:
WARNING: No metadata available for module 'utils': Unable to fetch interface description for command ''.

Details:
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\Users\Steven Pawley\AppData\Roaming\GRASS7\addons\docs\html\raster.html'

neteler · 2021-04-16T22:41:18Z

If I'm not mistaken, this worked in the past ... since when is it broken on Windows?

stevenpawley · 2021-04-19T05:57:06Z

I think it was working although it's a long time since I installed some of the modules fresh on Windows. I've tried to look through some of the changes with g.extension over the past few months but unfortunately, my understanding of how the makefiles are used is fairly poor :(

neteler · 2021-04-20T06:29:22Z

not sure if it's the same in linux?

While i.sentinel, i.fusion.hpf and r.learn.ml2 are installed successfully on Linux, in r.green are simply Python bug(s):

g.extension r.green
Fetching <r.green> from GRASS GIS Addons repository (be patient)...
Compiling...
Sorry: IndentationError: expected an indented block (plant.py, line 136)
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Python.make:5: /home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/etc/r.green/libhydro/plant.pyc] Error 1
Traceback (most recent call last):
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.theoretical", line 108, in <module>
    from libhydro.plant import power2energy
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/plant.py", line 136
    if elev is None:
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.theoretical.tmp.html] Error 1
Traceback (most recent call last):
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.planning", line 215, in <module>
    from libhydro.plant import power2energy
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/plant.py", line 136
    if elev is None:
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.planning.tmp.html] Error 1
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.technical", line 389
    return velocity**2 / (ks**2 * Rh**(4./3.)) * length
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.technical.tmp.html] Error 1
WARNING: You should install numexpr to use this module: pip install numexpr
Traceback (most recent call last):
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.optimal", line 114, in <module>
    from libhydro.optimal import find_segments
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/optimal.py", line 24, in <module>
    from libhydro.plant import COLS, COLS_points, Intake, Plant, Restitution
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/plant.py", line 136
    if elev is None:
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.optimal.tmp.html] Error 1
Traceback (most recent call last):
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.structure", line 144, in <module>
    from libhydro.optimal import conv_segpoints
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/optimal.py", line 24, in <module>
    from libhydro.plant import COLS, COLS_points, Intake, Plant, Restitution
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/plant.py", line 136
    if elev is None:
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.structure.tmp.html] Error 1
Traceback (most recent call last):
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.hydro.delplants", line 116, in <module>
    from libhydro.plant import read_plants, write_plants
  File "/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/r.green.hydro/libhydro/plant.py", line 136
    if elev is None:
    ^
IndentationError: expected an indented block
make[2]: *** [/home/mneteler/software/grass78/dist.x86_64-pc-linux-gnu/include/Make/Html.make:14: r.green.hydro.delplants.tmp.html] Error 1
/home/mneteler/tmp/grass7-mneteler-718443/tmpe_hh9zux/r.green/scripts/r.green.install:35: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Installing...

neteler · 2021-04-20T07:27:49Z

... in r.green are simply Python bug(s):

Addressed in OSGeo/grass-addons#513, pls check.

hellik · 2021-04-20T19:13:59Z

While i.sentinel, i.fusion.hpf and r.learn.ml2 are installed successfully on Linux

r.learn.ml2 installs in windows too; though there is again the message of a missing metadata and html of stats.py, which is a lib of r.learn.ml2:

g.extension extension=r.learn.ml2                                               
Downloading precompiled GRASS Addons <r.learn.ml2>...
Fetching <r.learn.ml2> from <http://wingrass.fsv.cvut.cz/grass78/x86_64/addons/grass-7.8.5/r.learn.ml2.zip> (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'stats'.   <==
WARNING: No metadata available for module 'r.learn.predict'.
WARNING: No metadata available for module 'indexing'.
WARNING: No metadata available for module 'utils'.
WARNING: No metadata available for module 'transformers'.
WARNING: No metadata available for module 'raster'.
WARNING: No metadata available for module 'r.learn.train'.
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\\Users\\yourusername\\AppData\\Roaming\\GRASS7\\addons\\docs\\html\\stats.html'    <==

it seems winGRASS g.extension expects a manual of the stats.py helper lib which has no manual as it is a lib script.

hellik · 2021-04-20T19:23:41Z

While i.sentinel, i.fusion.hpf and r.learn.ml2 are installed successfully on Linux

regarding i.sentinel, following scripts are installed:

C:\Users\yourname\AppData\Roaming\GRASS7\addons\scripts>dir /b
i.sentinel.coverage.py
i.sentinel.download.py
i.sentinel.import.py
i.sentinel.mask.py
i.sentinel.parallel.download.py
i.sentinel.preproc.py

though I get a message of a missing manual and metadata:

g.extension extension=i.sentinel                                                
Downloading precompiled GRASS Addons <i.sentinel>...
Fetching <i.sentinel> from <http://wingrass.fsv.cvut.cz/grass78/x86_64/addons/grass-7.8.5/i.sentinel.zip> (be patient)...
Updating extensions metadata file...
Updating extension modules metadata file...
WARNING: No metadata available for module 'i.sentinel.download'.  <==
ERROR: Unable to read manual page: [Errno 2] No such file or directory: 'C:\\Users\\youruser\\AppData\\Roaming\\GRASS7\\addons\\docs\\html\\i.sentinel.download.html' <==

there is no i.sentinel.download.html in the prepared addon zip file

C:\Users\youruser\AppData\Roaming\GRASS7\addons\docs\html>dir /b | grep html
i.sentinel.coverage.html
i.sentinel.html
i.sentinel.import.html
i.sentinel.mask.html
i.sentinel.parallel.download.html
i.sentinel.preproc.html

hellik · 2021-04-20T19:44:59Z

and the i.landsat-addon suite is missing in the prebuilt winGRASS addons via g.extension, see

although the addon suite is prepared, see

https://wingrass.fsv.cvut.cz/grass78/x86_64/addons/latest/?C=N;O=A
https://wingrass.fsv.cvut.cz/grass78/x86_64/addons/grass-7.8.5/?C=N;O=A

@landam any idea why it's missing in the g.extension download, although the zip is on the server

ninsbl · 2021-05-14T21:52:39Z

#1565 needs to be backported...

neteler · 2021-05-14T23:14:42Z

#1565 needs to be backported...

Done in #1575

hellik · 2021-05-16T08:43:28Z

beside the lazy import issues in python addons (e.g. r.green, i.sentinel.download, etc) to build the manuals, the (nested) addons with helper libs issue is solved. closing ticket

nilason · 2021-05-16T09:18:45Z

beside the lazy import issues in python addons (e.g. r.green, i.sentinel.download, etc) to build the manuals, the (nested) addons with helper libs issue is solved. closing ticket

Great!

hellik added bug Something isn't working Windows Microsoft Windows specific labels Apr 3, 2021

neteler mentioned this issue Apr 20, 2021

r.green: Python, Makefile and manual fixes OSGeo/grass-addons#513

Merged

ninsbl mentioned this issue May 6, 2021

g.extension: match module files on Windows #1565

Merged

neteler mentioned this issue May 8, 2021

g.extension/Windows: fix special characters OSGeo/grass-addons#526

Closed

hellik closed this as completed May 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] warning and error messages by g.extension while installing (nested) addons with helper libs #1499

[Bug] warning and error messages by g.extension while installing (nested) addons with helper libs #1499

hellik commented Apr 3, 2021

hellik commented Apr 3, 2021

veroandreo commented Apr 3, 2021

stevenpawley commented Apr 16, 2021

neteler commented Apr 16, 2021

stevenpawley commented Apr 19, 2021

neteler commented Apr 20, 2021 •

edited

neteler commented Apr 20, 2021

hellik commented Apr 20, 2021

hellik commented Apr 20, 2021 •

edited

hellik commented Apr 20, 2021

ninsbl commented May 14, 2021

neteler commented May 14, 2021 •

edited

hellik commented May 16, 2021

nilason commented May 16, 2021

[Bug] warning and error messages by g.extension while installing (nested) addons with helper libs #1499

[Bug] warning and error messages by g.extension while installing (nested) addons with helper libs #1499

Comments

hellik commented Apr 3, 2021

hellik commented Apr 3, 2021

veroandreo commented Apr 3, 2021

stevenpawley commented Apr 16, 2021

neteler commented Apr 16, 2021

stevenpawley commented Apr 19, 2021

neteler commented Apr 20, 2021 • edited

neteler commented Apr 20, 2021

hellik commented Apr 20, 2021

hellik commented Apr 20, 2021 • edited

hellik commented Apr 20, 2021

ninsbl commented May 14, 2021

neteler commented May 14, 2021 • edited

hellik commented May 16, 2021

nilason commented May 16, 2021

neteler commented Apr 20, 2021 •

edited

hellik commented Apr 20, 2021 •

edited

neteler commented May 14, 2021 •

edited