Replace "config_cache.xml" with internal python data structure?

[nusbaume]

The file "config_cache.xml" appears to only be used by the "build-namelist" perl script to generate the CAM namelist. This could be done instead by simply having "buildcpp" output a python data structure (like an object or list) directly to "buildnml", which could then use that information as-is, by-passing the need to read/write an XML file.

Of course, the dis-advantage is that the "config_cache.xml" file does potentially provide additional information/meta-data to users, which may be hidden if everything is passed in python internally. This will also require "buildcpp" to be run every-time "buildnml" or "build-namelist" is run, which could make certain script calls by the user take longer.

Does anyone have any thoughts or opinions on this? I figured I would leave this as just a pure discussion for now, as it isn't "required" to get CAMDEN up and running, although it might be easier to implement sooner rather than later.

[gold2718]

I would also like some feedback and thinking. Two quick thoughts.

1. The code that produces the configuration object from the case object could be a separate function we could call both from buildcpp and from buildnml. This would be fairly quick and would allow for more information such as information that has changed since buildcpp has been run.
2. Can you think of any information/metadata which would not be available using this method?

[nusbaume]

Thanks for the feedback! In response to your thoughts:

1. I am happy with creating a new function for this, and in fact if we do I believe we won't even need to call it in buildcpp, as buildnml is the only script that actually needs the variables.

2. I don't think any information would be lost, I just wonder if we should also create some sort of special query or print function that provides the variable descriptions and data currently contained in "config_cache.xml" to the user (say by printing it on the screen).

Arguably this query or print function isn't necessary at all, as the user can always find the relevant information elsewhere (like from the actual namelist). However, I don't know if there are users who look at the "config_cache.xml" file frequently when they work with the model, and if so I would ideally like to impact their personal workflow as little as possible by providing an alternative method for determining configure variable values and meta-data.

Anyways, just my current thoughts on the subject.

[gold2718]

Good call on the output. I suggest that we use CIME's logger functionality. The Class should have a print or log method. I would start out with logger.debug (called from buildnml) and see if anyone wants it bumped up to logger.info after it is used for a while.

[cacraigucar]

I would want to be able to investigate the config_cache.xml-like data. I don't look at it frequently, but when I need it, it is very valuable.

…

On Thu, Aug 22, 2019 at 9:00 PM goldy ***@***.***> wrote: Good call on the output. I suggest that we use CIME's logger functionality. The Class should have a print or log method. I would start out with logger.debug (called from buildnml) and see if anyone wants it bumped up to logger.info after it is used for a while. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACRX2X7ZEQFEU7J26YV4ENDQF5HDJANCNFSM4IOOFOTA> .

-- Cheryl Craig cacraig@ucar.edu NCAR Voice: (303) 497-2901 P.O. Box 3000 Fax: (303) 497-1324 Boulder, CO 80307

[gold2718]

@cacraigucar, this is why I want a logging / print option (see last bullet group below for how you could use this). Here is one idea of how to pursue this (and things to think about):

• Create a new file in cime_config which holds the Class which holds all the information which can affect the CAM build or namelist.
• The Class will be able to output to the log and to print (more details on how to do that below).
• When the Class is initialized with a case object, it will fill its contents with information from the case.
• The Class could have a log_or_print method which will log and/or print a message depending on inputs. The messages should be formatted for readability.
• The __init__ method will collect all the necessary information from the case and log each item (debug level) but not print.
• One responsibility of the Class is to be able to return a dictionary to the buildnml script for use in generating the correct default values. Depending on the design of buildnml, the Class may need to also provide other information (e.g., location of distributed namelist files to include).
• One responsibility of the Class is to be able to provide any information needed to configure the build.
• If this module is called from the command line, it will operate in one of these modes:
  • With no arguments, it will parse the current directory and print the values. If the current directory is not a CASEROOT, raise an Exception.
  • With one argument (a directory), parse and print the case values from that directory (with the same error processing).
  • With two arguments (directories), parse both directories and print the values which are different.

@cacraigucar, Would this sort of functionality meet your needs? I think the comparison mode could be especially useful but in any case, you could always just direct output to a file. The output should be much more readable than an XML file.

[cacraigucar]

That seems to do what I'd need. I don't need the two argument functionality because if I wanted to do that, I could easily capture my two outputs and diff them (assuming the ordering doesn't change between them). I'd want to be able to do an xxdiff, so no special characters which trick it into thinking its a binary file, please.

…

On Fri, Aug 23, 2019 at 7:43 PM goldy ***@***.***> wrote: @cacraigucar <https://github.com/cacraigucar>, this is why I want a logging / print option (see last bullet group below for how you could use this). Here is one idea of how to pursue this (and things to think about): - Create a new file in cime_config which holds the Class which holds all the information which can affect the CAM build or namelist. - The Class will be able to output to the log and to print (more details on how to do that below). - When the Class is initialized with a case object, it will fill its contents with information from the case. - The Class could have a log_or_print method which will log and/or print a message depending on inputs. The messages should be formatted for readability. - The __init__ method will collect all the necessary information from the case and log each item (debug level) but not print. - One responsibility of the Class is to be able to return a dictionary to the buildnml script for use in generating the correct default values. Depending on the design of buildnml, the Class may need to also provide other information (e.g., location of distributed namelist files to include). - One responsibility of the Class is to be able to provide any information needed to configure the build. - If this module is called from the command line, it will operate in one of these modes: - With no arguments, it will parse the current directory and *print* the values. If the current directory is not a CASEROOT, raise an Exception. - With one argument (a directory), parse and *print* the case values from that directory (with the same error processing). - With two arguments (directories), parse both directories and print the values which are different. @cacraigucar <https://github.com/cacraigucar>, Would this sort of functionality meet your needs? I think the comparison mode could be especially useful but in any case, you could always just direct output to a file. The output should be much more readable than an XML file. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACRX2X2TP4FRVGRMAUCCRXLQGCG53ANCNFSM4IOOFOTA> .

-- Cheryl Craig cacraig@ucar.edu NCAR Voice: (303) 497-2901 P.O. Box 3000 Fax: (303) 497-1324 Boulder, CO 80307