raster module memory: set new value globally #922
Conversation
This change allows to globally override the default 300 MB with a user defined value:
r.in.gdal --help
Imports raster data into a GRASS raster map using GDAL library.
Usage:
r.in.gdal [-ojeflakcrp] input=name output=name
...
Parameters:
input Name of raster file to be imported
output Name for output raster map
band Band(s) to select (default is all bands)
memory Maximum memory to be used (in MB)
default: 300
target Name of GCPs target location
title Title for resultant raster map
offset Offset to be added to band numbers
default: 0
g.gisenv set="MEMORYMB=6000"
r.in.gdal --help
Imports raster data into a GRASS raster map using GDAL library.
Usage:
r.in.gdal [-ojeflakcrp] input=name output=name
...
Parameters:
input Name of raster file to be imported
output Name for output raster map
band Band(s) to select (default is all bands)
memory Maximum memory to be used (in MB)
default: 6000
target Name of GCPs target location
title Title for resultant raster map
offset Offset to be added to band numbers
default: 0
g.gisenv set="MEMORYMB=300"
Based upon demo patch my @metzm
There was a problem hiding this comment.
a) Compiler warning (casting needed?):
main.c:195:12: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] 195 | memstr = G_getenv_nofatal("MEMORYMB");
C, GCC and/or Linux distros now take a lot of warnings more seriously than before, so we have a lot of these issues now.
In this case, defining memstr as const char* instead of char* should help. memstr can change, only the pointed to string cannot. (G_getenv_nofatal gives read-only pointers to an internal env data structure.)
b) How to do the same for the Python scripts?
$ ag 'answer: 300' scripts/r.fillnulls/r.fillnulls.py 107:#% answer: 300
In theory, you could hack parser, that's a bad option, or you could make the interface definitions in Python dynamic as they are in C; that would be great, but I guess too much.
I think the best way to go about it is to define a standard option. This will move the default to the C level. It should work similarly to C only the module won't be able to set its own default which is possible in C with the current implementation.
Clearly, there is already a couple of those around, so standard option makes sense. It should also greatly reduce the duplication in your C code.
|
Thanks for your review @wenzeslaus !
ok I hope I got it right (the updated C code works and the compiler no longer complains (see lib/gis/parser_standard_options.c in this PR).
Good suggestion, I have updated the PR accordingly and moved it all to While it should be all done for the C modules, it is yet not liked by Python: e.g. (see the CI): Any pointers here? |
|
Remaining error, I don't find where it comes from: (from CI) |
|
The manuals are created at compile time, requiring a default answer to options. With your PR, the default answer can be changed after compile time, at runtime. This PR adjusts the default answer at runtime, but there is no default answer at compile time. index d8dba10d5..a0065a11a 100644
--- a/lib/gis/parser_standard_options.c
+++ b/lib/gis/parser_standard_options.c
@@ -138,7 +138,7 @@
struct Option *G_define_standard_option(int opt)
{
struct Option *Opt;
- const char *memstr;
+ char *memstr;
Opt = G_define_option();
@@ -253,11 +253,13 @@ struct Option *G_define_standard_option(int opt)
Opt->key_desc = "memory in MB";
Opt->required = NO;
Opt->multiple = NO;
- /* first check MEMORYMB in GISRC, set with g.gisenv */
- memstr = G_getenv_nofatal("MEMORYMB");
- if (!memstr)
- memstr = "300";
- Opt->answer = (char *) memstr;
+ Opt->answer = "300";
+ /* start dynamic answer */
+ /* check MEMORYMB in GISRC, set with g.gisenv */
+ memstr = G_store(G_getenv_nofatal("MEMORYMB"));
+ if (memstr && *memstr)
+ Opt->answer = memstr;
+ /* end dynamic answer */
Opt->label = _("Maximum memory to be used (in MB)");
Opt->description = _("Cache size for raster rows");
break;
diff --git a/man/parser_standard_options.py b/man/parser_standard_options.py
index 5e2c753a5..6558bf421 100644
--- a/man/parser_standard_options.py
+++ b/man/parser_standard_options.py
@@ -50,7 +50,14 @@ def parse_options(lines, startswith='Opt'):
def parse_glines(glines):
res = {}
key = None
+ dynamic_answer = False
for line in glines:
+ if line.strip() == "/* start dynamic answer */":
+ dynamic_answer = True
+ if line.strip() == "/* end dynamic answer */":
+ dynamic_answer = False
+ if dynamic_answer or line.startswith('/*'):
+ continue
if line.startswith('/*'):
continue
if line.startswith(startswith) and line.endswith(';'):This is a hard-coded hack to enable creation of manual pages. |
- fixes the creation of man pages #922 (comment) Contributed by @metzm
Ah, that's explaining it.
Thanks so much, I have added your patch in 69367ff
... which is acceptable as MEMORYMB changing during compile time being rather unlikely, IMO. |
|
The "Changes requested" isn't valid any more but I have no idea how to reset it... |
* raster module memory: set other value globally
This change allows to globally override the default 300 MB with a user defined value:
r.in.gdal --help
Imports raster data into a GRASS raster map using GDAL library.
Usage:
r.in.gdal [-ojeflakcrp] input=name output=name
...
Parameters:
input Name of raster file to be imported
output Name for output raster map
band Band(s) to select (default is all bands)
memory Maximum memory to be used (in MB)
default: 300
target Name of GCPs target location
title Title for resultant raster map
offset Offset to be added to band numbers
default: 0
g.gisenv set="MEMORYMB=6000"
r.in.gdal --help
Imports raster data into a GRASS raster map using GDAL library.
Usage:
r.in.gdal [-ojeflakcrp] input=name output=name
...
Parameters:
input Name of raster file to be imported
output Name for output raster map
band Band(s) to select (default is all bands)
memory Maximum memory to be used (in MB)
default: 6000
target Name of GCPs target location
title Title for resultant raster map
offset Offset to be added to band numbers
default: 0
g.gisenv set="MEMORYMB=300"
Based upon demo patch my @metzm
* fix double description
* code simplification by defining new G_define_standard_option(G_OPT_MEMORYMB)
* update STD_OPT_STRINGS after include/gis.h modification
* Patch added to also set default answer at compile time
* document new GRASS variable MEMORYMB
- fixes the creation of man pages
#922 (comment)
Contributed by @metzm
* This adds an NPROCS GIS environment variable. * This change allows to globally override the default value (1) with a user defined value for the nprocs option using g.gisenv. * This follows the existing implementation for memory with MEMORYMB variable (PR #922). * nprocs option already exists, so this adds only the variable NPROCS and its documentation.
* This adds an NPROCS GIS environment variable. * This change allows to globally override the default value (1) with a user defined value for the nprocs option using g.gisenv. * This follows the existing implementation for memory with MEMORYMB variable (PR OSGeo#922). * nprocs option already exists, so this adds only the variable NPROCS and its documentation.
* This adds an NPROCS GIS environment variable. * This change allows to globally override the default value (1) with a user defined value for the nprocs option using g.gisenv. * This follows the existing implementation for memory with MEMORYMB variable (PR OSGeo#922). * nprocs option already exists, so this adds only the variable NPROCS and its documentation.
This change allows to globally override (as being written to
$HOME/.grass7/rc) the default 300 MB raster cache setting with a user defined value:Based on demo patch my @metzm - thanks!
Open issues:
a) Compiler warning (casting needed?):
b) How to do the same for the Python scripts?