Collecting feedback on the webpage created by ADF

[cecilehannay]

Original request (02/22/2022)

Cecile started to run the ADF for CAM7 development runs in addition to the old AMWG diags.

The first run to have ADF diags can be find at: NCAR/amwg_dev#78
For that run, here are the ADF diags versus AMWG diags

Cecile will encourage scientists to look at the diags during the CAM7 meeting. We expect they will have feedback. For instance, do they need variables to be grouped by categories (surface variable, TOA variables, ...) like in the old diags or could they live without? What would be the most useful way to group the diags.

I am adding @JulioTBacmeister to the discussion. As the amwg co-chair, he will want to be part of the discussion too.

Update 04/02/2022

There has been a lot of replies and I am trying to provide a summary of what has been discussed so far:

• we will start with Dani's prototype. Justin will take over the webpage design.
• the variables will appear by categories as defined in this spreadsheet (user_nl_cam is also available here). The categories are for the model to model comparison. We will have to give more thoughts about how we want to organize categories for the model to obs.
• for easy navigation, the variables will appear as a list of links on the left side and the plots will show on the right side (similar to Rich's SCAM diags except the variables will be on the left column)
• We will create a new github repo for scripts/notebooks provided by the community but that don't fit in the ADF framework (for instance Dan's widget).

More suggestions:

• adding colored tables for the model to model comparison with red/green to show which run of the two is better compared to obs (the land does something like this and we had some simple tables doing this in the orig. amwg diags)
• introducing a pattern correlation to the top of the anomaly map plots, in addition to the mean and rmse (similar to ncl pattern_corsubroutine)

Update: 06-24-2022

The present discussion is now closed and is replaced by: #189

[andrewgettelman]

Thanks Cecile for raising the issue and starting the discussion.

I do think we should have an option for grouping variables by 'category'. I think we do not need to group by plot types on separate page as well.

Mechanistically: we could add a 'category' attribute to the variable definitions file and have the code sort on the categories it finds, and order the web page that way? Just a thought.

[dan800]

Are their ADF diags for height resolved fields? I couldn't see them.

[andrewgettelman]

@dan800 Zonal mean plots work with 2-D and 3-D fields (latitude-height).

[dan800]

@andrewgettelman As far as I can see, all fields for the ADF diags listed below are not height resolved - please point me to the page in the ADF link above for zonal mean T, U, V, etc., I can't find it - it looks like they were not produced.

[andrewgettelman]

Dan: remember ADF is a more flexible framework than the AMWG package: not everything possible was implemented in that particular run. Think of it as a sample.

If you give the ADF zonal mean plots a 3D field, it will produce a latitude-height plot.

[dan800]

Yes, I am well aware - we were asked to give feedback on this set of diags above. My feedback is they are not particularly useful to me unless they have height resolved fields as default, but before I said that I wanted to make sure I wasn't missing them in some link somewhere. Apologies if it wasn't clear why I was asking for the height resolved fields.

[andrewgettelman]

The discussion here I believe is for organizing the web page. There seems to be a need to group variables by 'category' as noted above. Right now all plot types are listed by variable, not as separate pages as in the old AMWG diagnostics. General question is whether that is okay or not?

I think I would propose that we start with categories and then see if we need to break out by plot type too.

[dan800]

Thoughts on categories -- Whichever way we subdivide we are likely going to run into a situation where we have to traverse down several trees to get plots next to each other that can answer science questions. e.g., if you divide by field at the top of the tree and you may want to look at T and H2O zonal means vs height; divided by plot type and you may want to compare ozone column to height resolved; divide by physical process (e.g, energetics, chemistry) you may want to compare a heating rate and a constituent. A possible solution to this is that a future interface allows filtering of plots - e.g., filter by plot type - lat only, lat/lon, lon/P, species/variable (all T plots eg), category (microphysics only, chem only, upper atmosphere only).
But I think to begin with by plot/table type makes the most sense, which is sor of what the old diags did.

[JulioTBacmeister]

I think we need a default set of lat-lon map plots using categories something like those used in the current AMWG diags: roughly - clouds, radiation, surface fluxes, winds/dynamics. Then we should also have two default sets of zonal mean plots: one focused on the troposphere, i.e, surface to 10-15km; and another going to the top of the model. These should have at least U,T,Q, RelHum and depending on the type of run maybe some chemical species such as CH4, N2O.

[brianpm]

@JulioTBacmeister -- Just to be explicit, are you suggesting that the hierarchical structure of the web interface to the ADF plots should look like this:

lat-lon map plots

clouds

• CLDLOW ANN DJF MAM JJA SON
• CLDMED ANN DJF MAM JJA SON
• CLDHGH ANN DJF MAM JJA SON
• SWCF ANN DJF MAM JJA SON
• LWCF ANN DJF MAM JJA SON

radiation

• FSNT ANN DJF MAM JJA SON
• FLUT ANN DJF MAM JJA SON
• FSNTC ANN DJF MAM JJA SON
• FLUTC ANN DJF MAM JJA SON
• FSNS ANN DJF MAM JJA SON
• FSNSC ANN DJF MAM JJA SON
• FLDS ANN DJF MAM JJA SON
• FLDSC ANN DJF MAM JJA SON

surface fluxes

• LHFLX ANN DJF MAM JJA SON
• SHFLX ANN DJF MAM JJA SON
• PRECT ANN DJF MAM JJA SON
• PRECC ANN DJF MAM JJA SON
• PRECL ANN DJF MAM JJA SON

winds/dynamics

• wind vectors @ 200hPa ANN DJF MAM JJA SON
• wind vectors @ 850hPa ANN DJF MAM JJA SON

zonal mean plots

surface to 10-15km {linear in pressure}

• T ANN DJF MAM JJA SON
• U ANN DJF MAM JJA SON
• Q ANN DJF MAM JJA SON
• RELHUM ANN DJF MAM JJA SON
• CH4 ANN DJF MAM JJA SON
• N2O ANN DJF MAM JJA SON

to the top of the model {log pressure}

• T ANN DJF MAM JJA SON
• U ANN DJF MAM JJA SON
• Q ANN DJF MAM JJA SON
• RELHUM ANN DJF MAM JJA SON
• CH4 ANN DJF MAM JJA SON
• N2O ANN DJF MAM JJA SON

[juliecaron]

i mostly like this. i can see including a surface variables section in the lat/lon plot-type containing precipitation, SST, PSL or the like ( did we include the precipitation variables in a flux section in amwg diags? maybe that would be more like P-E and precip would go under sfc vars). and we should have a section for lat/lon vector plots as well

[JulioTBacmeister]

I like this basic structure. I think we should have PSL and surface windstress somewhere. My inclination would be to add windstress to the "flux" category and put PSL/PS in the lat-lon dynamics plots.

[cecilehannay]

Some Practical Considerations (discussed at the CAM7 meeting)

• Website generation needs to be flexible enough to deal with having more/less variables. Hard-coding the HTML is not a good idea.
• End result is a static website. This is required for portability: run ADF on casper, move website to server or local machine.
• We have NO WEB DEVELOPER involved.
  • Web site generator uses a simple templating engine and very basic HTML/CSS.
  • Adding functionality (interactivity, conditional selection, etc.) is not feasible unless we bring in someone else.

[cecilehannay]

@bitterbark
The plots are here:
/glade/work/hannay/ADF/plots/f.e21.FWscHIST.ne30_L48_BL10_cam6_3_041_kzz3_zmtop75.hf.001_vs_f.e21.FWscHIST.ne30_L48_BL10_cam6_3_041_control.hf.001_1980_1989

For that run, here are the ADF diags versus AMWG diags

[dan800]

Hello all,

Concerns about having no web developer (and @brianpm's weekends!) led me to think about other ways we could approach viewing the output from the diagnostics. If we split the ADF into a part to create 'assets' (i.e., the pngs) from how they are viewed we might be able to have a variety of ways to view them. There are probably many ways to create an asset browser, and doing so might be a standalone activity that would be relatively easy to have the broader community help co-develop. One approach that would work for in-house CAM7 development right now is to view them from a jupyter notebook running on cheyenne or casper. It could also run on a local laptop when accessing pngs from the cgd web server.

I shared a very basic notebook with Brian and Cecile which was able to access the pngs in the asset directory of the ADF diags. It simply provides drop down menus for the field, season and plot type.

The code is short and simple and can be found here:
/glade/u/home/marsh/projects/CAM_diagnostics/notebooks
ADF_widget.ipynb - super simple notebook that parses the contents of the asset dir to create the menu labels
ADFAB.ipynb - an even simpler one that reads in a json file of metadata on the contents of the asset dir to make the browser interface
ADF_assets.json - the metadata on the contents of the asset directory - ideally this would reside in the asset dir
create_asset_metadata.ipynb - routine to create the metadata json file - could be integrated into ADF
If you copy it over to a local directory on glade it should work. Please understand this was an afternoon's work so more of a proof of concept.

One issue is if people would be willing to go the extra step of opening up a notebook. There are ways to make notebooks web accessible - e.g., send it to github and use github actions to generate a jupyter book. Brian suggested looking into Voila.

[aside] It would be good if the ADF could make parsing the filenames easier. Right now the delimiter is '_' but it's also mixed into the variable names - so I had to skip the variables that included an _ and hard wire plot types. Keeping an underscore for the variable names and using a '.' might make it simpler - e.g., STEND_CLUBB_ANN_Zonal_Mean.png -> STEND_CLUBB.ANN.Zonal_Mean.png, so string splitting is always {variable}.{time averaging}.{plot type}.{file format}
[/aside]

[andrewgettelman]

Thanks very much for taking a crack at this Dan and pushing it forward. I like this idea Dan as another optional entry point to the plots that may be useful to people, and opportunities for developing another visualization workflow using the existing plots.

I don't see a problem with the change in file names. Doesn't seem like it would be too hard to fix if you know where to look in the code.

[dan800]

Hi Andrew,

I think these are the relevant lines:

scripts/plotting/global_latlon_map.py:                            plot_name = plot_loc / "{}_{}_LatLon_Mean.{}".format(var, s, plot_type)
scripts/plotting/zonal_mean.py:                    plot_name = plot_loc / "{}_{}_Zonal_Mean.{}".format(var, s, plot_type)
scripts/plotting/polar_map.py:                            nh_plot_name = plot_loc / f"{var}_{s}_NHPolar_Mean.{plot_type}"
scripts/plotting/polar_map.py:                            sh_plot_name = plot_loc / f"{var}_{s}_SHPolar_Mean.{plot_type}"

I think these would be the lines to change from _ to . (except for the one before the word Mean)
But this would likely break your current webpage generator.

[bitterbark]

Static webpage example
to aid in discussion (separate from Jupyter lab idea)

[cecilehannay]

@bitterbark: Thanks for preparing this. People liked the page overall.

There were a few suggestions that came up. Look also at Rich's post below.

[cecilehannay]

From @swrneale (I am reposting here to make sure people see it):

My comments would be:
-Have the plot show on the right side while the list of links remains on the left
-Maybe have a 'top ten' grouping so that we can quick look at the most frequent plots (PRECT, SWCF, LWCF,TS, etc.)
-Colorizing the tables red/green to show which run of the two is better compared to obs., the land do something like this and we had some simple tables doing this in the orig. diags.
-In the 'links' it would be good to be able to go to other milestone diagnostics (CAM6/CESM2/20th/FHIST etc.)
Looking good though!

Simple example of multi-panel view of figures using SCAM.
http://wwwint.cgd.ucar.edu/project/amp/rneale/scam/web/cesm2_zm01-cesm2_zm00-ARM95/index.html

Rich

[cecilehannay]

There are all great suggestions, @swrneale.

• I especially like the colorized table. (I was always envious of the land)
• And also the multi-panel view of the figures suggestion. It would be a way to address Dan's comment to easily navigate from one plot to the other.

[cecilehannay]

justin-richling
Can you please point me to an example of a colorized table you were mentioning earlier in the CAM7 meeting?

swrneale
The old diagnostics package creates an ncl table and displays a png of that on the webpage. It is also possible to colorize the background of each cell in html on the table webpage, but I think the best option is to create the actual table in python and then display that as a png, this gives us more flexibility to do other things I think. There are a few packages to export a stylized pandas dataframe to an image e.g.,(https://pypi.org/project/dataframe-image/)

adamrher
I'm wondering if we should introduce a pattern correlation to the top of the anomaly map plots, in addition to the mean and rmse? I've found it a useful metric for evaluating the skill of my CLUBB+MF runs.

I use the ncl routine pattern_cor: https://www.ncl.ucar.edu/Document/Functions/Contributed/pattern_cor.shtml. I'm sure there's a python library with it out there somewhere.

[cecilehannay]

There has been a lot of replies and I am trying to provide a summary of what has been discussed so far:

• we will start with Dani's prototype. Justin will take over the webpage design.
• the variables will appear by categories as defined in this spreadsheet (user_nl_cam is also available here). The categories are for the model to model comparison. We will have to give more thoughts about how we want to organize categories for the model to obs.
• for easy navigation, the variables will appear as a list of links on the left side and the plots will show on the right side (similar to Rich's SCAM diags except the variables will be on the left column)
• We will create a new github repo for scripts/notebooks provided by the community but that don't git in the ADF framework (for instance Dan's widget).

More suggestions:

• adding colored tables for the model to model comparison with red/green to show which run of the two is better compared to obs (the land does something like this and we had some simple tables doing this in the orig. amwg diags)
• introducing a pattern correlation to the top of the anomaly map plots, in addition to the mean and rmse (similar to ncl pattern_corsubroutine)

[cecilehannay]

The present discussion is now closed and is replaced by: #189