PPDG_Add_Diags

AMWG Diagnostics – Developer Hints

C. Bardeen & J. Dowling July 25, 2016

This summarizes our experience trying to extend the AWMG diagnostics. This may help you get started, but beware that things may have changed (hopefully for the better) and your mileage may vary. This document assumes that you already know how to run the diagnostics, and are familiar with NCL and HTML.

There are several common ways that you might want to extend the AWMG diagnostics packages, including:

• Adding a new diagnostic set
• Adding an observational data set to an existing diagnostic set
• Adding a new field to an existing diagnostic set
• Changing the layout of an existing html page for a diagnostic set
• Changing the format of a plot already in a diagnostic set

We will tackle these in reverse order, since they build upon each other. Please read all the items that precede the one you are interested in as those comments will probably also apply to your case.

In these descriptions, we will use one of the diagnostics sets that we added, wset3 (WAWG set 3), as an example.

Changing a plot in a diagnostic set

Each diagnostic set has one or more plotting routines that are called to generate each plot. Each plotting routine uses a set of environment variables set from the driver script to control how it operates. Typically each routine will be coded to perform differently when comparing with another model (COMPARE!=”OBS”) versus comparing to observations (COMPARE=”OBS”). To find out which script is being used for a plot, you can look in the python object for the diagnostic set. For wset 3, that is:

diagnostics/diagnostics/atm/Plots/wset3.py

If you look at the __init__ method, you will see the name(s) of the script(s) being set:

self.ncl_scripts = ['plot_waccm_vcycle.ncl']

You will find this function in the atm_diag directory. Each plotting script is different, so you will have to study the script if you wish to change its behavior. If you change the number of plots it makes or the names of the plots it makes, then you will need to modify the python class and the html. Otherwise, you can change whatever you want about a plot by just changing the plotting script.

Changing the layout of an html page

If you want to change the text or the organization of the html page for a diagnostic set, there are two sets of files to examine: model-model and model-obs. For wset3, these are in:

atm_diag/html/model-obs/wset3, and

atm_diag/html/model1-model2/wset3.

Within each directory, you will see 2 or more files that contain html fragments that are combined to create the html page. The file wset3_1.htm contains the html header, and wset3_2.htm contains the body. Typically the body is constructed as a large table with a link to each plot in a cell of the table. The HREF tag is modified by the python code when the pages are assembled depending on whether the plot file got created or not and replaces xxx with the actual plot type, so it is important that you follow the default style for the HREF tags. Otherwise, you are free to format the page any way that you want. NOTE: Currently the html is stored in two places:

atm_diag/html, and

diagnostics/diagnostics/atm/html

The intent is to keep these identical and eventually remove the ones in atm_diag/html; however, those are currently the ones that are operational.

Adding a new field to an existing diagnostic set

Typically each diagnostic set contains one type of plot, but if you want to add the same type of plot for another field, you will need to do 3 things:

1. Modify the plot script to add the extra variable that you want to plot. Each plot routine does this differently, but typically you will see a list of variables in the code and that list may be different for model-obs and model-model. It may be necessary select appropriate ranges, contour intervals, … in the plot script to get a meaningful plot.
2. The python object for the diagnostic set (e.g. wset3.py) wants to have a list of all the plot names that will be generated. So you need to modify the __init__ method to add the name of the new plots associated with your new variable.
3. You need to modify the html to add an HREF to your new plot(s). If your new variable will be in model-model and model-obs, then you need to modify both sets of html files.

Adding observational data to an existing diagnostic set

Adding a new observational set is much like adding a new field, except there probably going to be many new fields added and you only need to worry about the model-obs side of the html. You also need to get your new data set added to the data repository. You will need to contact Cecile Hannay to see about doing that. Please look at the existing sets before you add one. Your data may already be there, and if you do need to add data, you should try to follow the style, units, and naming conventions found in the existing observational sets. This will make it easier for you to add other people’s data to your plots and for them to add your data to their plots. To debug your new data set, you can place the files in a directory and change the path to the observational data set in your env_diag_atm.xml file. If the data set uses other observational sets, you will need copy or link to the other data sets. At a minimum, you should plan to add a climatology file averaged for each month, each season, and annually.

Adding a new diagnostic set

Now for the hard one. It is everything you have already done and a little bit more. Rather than just modifying files from an existing diagnostic set, you will need to:

• add xml settings for the new set
• create a python object for the set and add it to the class factory
• create the NCL plot script(s)
• create new html pages for the set

The best way to do most of this is to copy files from and existing diagnostic set that is close to what you want and then modify them for your purposes.

For the xml settings, you will need to modify diagnostics/diagnostics/atm/Config/config_diags_atm.xml and the env_diag_atm_xml in your case directory. You need to add the XML definition of your net diagnostic set, in config, and then you need to set it in env_diag_atm.xml. You will need to set it to True for your set to be generated. You need to add a new object for your diagnostic set. For example the one for wset3 is diagnostics/diagnostics/atm/Plots/wset3.py. You will also need to add the creation of that class in the factory class diagnostics/diagnostics/atm/Plots/atm_diags_plot_factory.py and list it in diagnostics/diagnostics/atm/Plots/__init__.py. You need to modify the __init__ method to generate the correct file list, and to define the plot script(s) that will be called. You also need to modify the check_prerequisites method to modify any environment variables that need to be set for your plot routine.

You may be able to use one of the existing plot routines for your purposes, but more likely you will copy one and modify it. This helps identify some of the environment variables that your script should use to control its behavior. There are also some files with shared routines (functions_xxx.ncl) that you can use to make developing your plots easier. Remember for the HTML pages that you need to provide them for both model-model and model-obs. They are done in a similar way, but are independent sets of files. In atm_diag/html/model-obs and atm_diag/html/model1-model2, you will need to create a directory names for your diagnostic set and add the two template html files. Copy them from another set, and customize them for your needs. You will also need to modify the sets2.htm in each branch to link to your new set, and provide a new image for you page in atm_diag/html/images. In atm_diags/html, you also need to add your set to setup_2models, setup_obs, and mvgifs2www. Remember that all these changes need to be duplicated in diagnostics/diagnostics/atm/html.

Follow this recipe, and you should be able to quickly add a new diagnostic set, particularly if you have already have plotting routines for your data. In some cases, you may find that you need to add new environment variables to control your script or you need new types of data sets not already provide. Contact Alice Bertini if this applies to you.