series_analysis in a complex use case

[mpm-meto]

I have been working on a complex use case which involves processing the stats for 10 variables using EnsembleStat and GenEnsProd. So far so good. I then wanted to add series_analysis to verify the ensemble means from GenEnsProd, and this is where things break down.

If I am honest, I have never really understood the RUNTIME_FREQ options, especially the appearance of the "all" in the output file names, which seem outside any user control. Here's what I've concluded. It would be good to get some feedback on this and ideas going forward.

1. You can't run it in a complex use case because of the aggregation step, but I have tried all the options and none of them do what I want. I couldn't get it to produce aggregated output for a single lead time without restricting the lead time sequence to a single lead time, and I couldn't get the output filename to give me the lead time in the output filename. I would want the file name to conform to a more conventional filename convention. The outcome is the same whether I run one lead time or more. I get one output file with _all.

2. The next thing I have noted is that the series_analysis output variables never contain the name of the source variable, e.g. T250. Thus, I have now concluded that SeriesAnalysis cannot be run with a FCST_VAR[n]_NAME and OBS_VAR[n]_NAME because it was not designed to cope with multiple input variables. At first I thought it might be combining variables but I don't think it is. From what I can gather it processes/loops over all of them, writes to file, and then overwrites with each new variable because the output variable names do not reflect the source variable. In my case, the output files I've looked at seem to only have the LAST (10th) variable in the file.

To summarise, I think I am clear that one can't run SeriesAnalysis as part of the complex use case with conventional looping. RUNTIME_FREQ options don't appear to offer what I need though, with ambiguous naming conventions. I was expecting to be able to keep lead times separate (which is something that seems quite normal to do) but I couldn't get that to give me a sensible filename. That may just be me not quite finding the right combo but I have tried! Downstream software can't differentiate between "all" and "all". I can't think of a situation where we would want multiple lead times processed together into one output stat. I would rather not have to have a script with separate loops for each lead time and variable creating config files before running them. It's a solution, but not a pretty one, and doesn't solve the file naming convention without doing some renaming. Having netcdf files which have output filename structure more like other tools, and possibly something that incorporates the aggregation start and end date as defined by the date sequence may be handy.

Others may have known this but SeriesAnalysis can't write stats for multiple variables in one pass/file (without overwriting). The source variable can only be found in the global attributes and that isn't ideal for downstream plotting applications/tools.

It may be handy to update the series_analysis documentation to clarify some of these points. I have poured over the RUNTIME_FREQ examples and whilst they illustrate the behaviour, they do not provide the link with how this translates to the output file name generation (which is shared with GridDiag). It might be good to make it clear that you can only process one variable at a time, until this can be resolved.

Adding the source variable name in the output filename (if opting for one file per variable) and also in the output variable names would be good for downstream processing and parsing.

[j-opatz]

Hi Marion,

And thank you for your insights on this subject. I wanted to clarify a couple of points, since Series-Analysis is a tool that actually can do quite a bit, but it often causes a little frustration to get it to work the way you intend.

First, it's important to learn how Series-Analysis determines what the series will be created over. As seen in the MET documentation, the tool will check the following list, in order, to determine which entry is the first with a greater than 1 value:

1. the number of forecast fields in the configuration file
2. the number of observation fields in the configuration file
3. the number of input forecast files
4. the number of input observation files

Note how "number of fields" are the first two checks. This means if you provide multiple forecast or observation variables, those variable fields will constitute the series, rather than a time series. This addresses your second finding,

...SeriesAnalysis cannot be run with a FCST_VAR[n]_NAME and OBS_VAR[n]_NAME because it was not designed to cope with multiple input variables.

It was designed to handle multiple variables, but will process them together as a series. If your intent is to have multiple variable fields processed across time, you will have to use separate SeriesAnalysis tool instances, which are described here.

Second, the behavior of Series-Analysis in METplus wrappers is a bit complex, as you have experienced and noted. It sounds like you've already reviewed the Runtime Frequency section of the documentation, but may have left with more questions than answers. Given that you are trying to create a series over 1 lead time, SERIES_ANALYSIS_RUNTIME_FREQ = RUN_ONCE_PER_LEAD should be the behavior you want. It will step through each lead time and try to find a series across the valid and initialization times within your time boundaries.

Let's take a look at the example in the documentation to understand this better.

Assume that your timing information is the following:

LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d%H
INIT_BEG = 2020101712
INIT_END = 2020101912
INIT_INCREMENT = 1d

LEAD_SEQ = 3H, 6H

There will be three days of interest, each with 2 lead times (valid 2020-10-17 at 15Z and 18Z , 2020-10-18 at 15Z and 18Z, and 2020-10-19 at 15Z and 18Z). Setting SERIES_ANALYSIS_RUNTIME_FREQ = RUN_ONCE_PER_LEAD means that we are interested in creating a series across all of the days of interest for the 3H lead, and a second series across all of the days of interest for the 6H lead. In MET, that looks like substituting '*' for the initialization and valid times, which effectively gathers all of the available times (in other words, all of the available files) into a series. So for the 3H lead, we would have the following initialization times/files (note that the V indicates the valid hour):

• I2020101712_F003_V15
• I2020101812_F003_V15
• I2020101912_F003_V15

And for the 6H lead, we would have the following initialization times/files:

• I2020101712_F006_V18
• I2020101812_F006_V18
• I2020101912_F006_V18

When you see "all" in the file/variable names, essentially you're seeing the '*' character where the tool was attempting to grab all files in the series.

As a note to the overwriting behavior, make sure that your output template naming scheme uses the appropriate key to distinguish one series from another. For example, if you are creating a series over the lead, you may use an output template like:

SERIES_ANALYSIS_OUTPUT_TEMPLATE = L{lead?fmt=%3H}_stats.nc

Which creates a 3 digit lead, in hours. If you are running multiple variables via multiple tool instances, you should be able to use the instance to create a specific name so that the sequence for one lead does not overwrite another. For example, a tool instance evaluating temperature, SeriesAnalysis(temp), could have

[temp]
SERIES_ANALYSIS_OUTPUT_TEMPLATE = temp_L{lead?fmt=%3H}_stats.nc

While the instance evaluating accumulated precipitation, SeriesAnalysis(precip), could look like

[precip]
SERIES_ANALYSIS_OUTPUT_TEMPLATE = precip_L{lead?fmt=%3H}_stats.nc

I understand your frustration of the file naming convention; beyond the prefix that a user can control, there is no way to control the timing information section of the file. However, I think there is a way to partially override the variable name created by MET: using set_attr_name under FCST_VAR1_OPTIONS should keep the output variable name to a minimal amount of additional material. For example, I ran Grid-Stat with the following variable name (it's output from a previous METplus run):

name       = "FCST_t2m_P850_FULL";

And the resulting output field in the netCDF was a mouthful:

float FCST_FCST_t2m_P850_FULL_P850_WEST;

but after adding set_attr_name = "TEST", it's shortened to the following:

float FCST_TEST_P850_WEST

If you want to get the file's variable field name to exactly match your string (i.e. just "TEST"), then you will need to run the file through a follow-up tool like Regrid-Data-Plane to override the default name:

regrid_data_plane grid_stat_000000L_20140915_000000V_pairs.nc grid_stat_000000L_20140915_000000V_pairs.nc testingNaming.nc -field 'name="FCST_TEST_P850_WEST"; level="(*,*)";' -name TEST

ncdump -h testingNaming.nc

dimensions:
	lat = 361 ;
	lon = 720 ;
variables:
	float lat(lat) ;
		lat:long_name = "latitude" ;
		lat:units = "degrees_north" ;
		lat:standard_name = "latitude" ;
	float lon(lon) ;
		lon:long_name = "longitude" ;
		lon:units = "degrees_east" ;
		lon:standard_name = "longitude" ;
	float TEST(lat, lon) ;
		TEST:coordinates = "lat lon" ;
		TEST:name = "TEST" ;
		TEST:long_name = "2-meter temperature at P850 at P850" ;
		TEST:level = "P850" ;
		TEST:units = "K" ;
		TEST:_FillValue = -9999.f ;
		TEST:init_time = "20140915_000000" ;
		TEST:init_time_ut = "1410739200" ;
		TEST:valid_time = "20140915_000000" ;
		TEST:valid_time_ut = "1410739200" ;

I am more than happy to get into more details on this topic with you in a future meeting once funding has cleared, as well as take any suggestions you might have for clearing up the ambiguity that still surrounds this tool's operation.