Initial documentation of the UGRID capability.

docs/Users_Guide/appendixH.rst

@@ -0,0 +1,63 @@
+.. _appendixH:
+
+*****************************
+Appendix H Unstructured Grids
+*****************************
+
+Introduction
+============
+
+METv12.0.0+ includes limited support for using datasets on unstructured grids (UGRIDs) in the PointStat and GridStat tools. This support includes verifying UGRID forecasts against point observations (PointStat), and verifying UGRID forecasts regridded to a structured grid against gridded observations on a structured grid (GridStat). The implementation of UGRID support in METv12.0.0+ is based on the UGRID Conventions (https://ugrid-conventions.github.io/ugrid-conventions/). These conventions define various elements for common UGRID topologies. Thus far, development to support datasets on unstructured grids has been driven by the LFRIc (“elfrick”) NWP model, and the Model for Prediction Across Scales (MPAS) NWP model. However, the support for unstructured grids was implemented in such a way that additional unstructured grids can be utilized, provided the topology can be related to the UGRID conventions.
+
+Unstructured Grid Files
+=======================
+
+Support for UGRID files in NetCDF format is provided. MET will attempt to auto-detect UGRID files. If the file is CF-compliant and the “Conventions” global attribute is “UGRID” or “MPAS”, or if the global attribute “mesh_spec” is present, MET recognizes the file as a UGRID file. If the user is not using a supported UGRID, or a UGRID file that does not contain the required global attributes for autodetection, the user should set the **file_type** configuration entry to **NETCDF_UGRID**. In some cases, the UGRID topology is provided in an ancillary file separate from forecast variables. This is controlled via the **ugrid_coordinates_file** configuration entry. This file is checked first, and then the UGRID data file. If the same elements are found in both files, the information from the data file takes precedence, and thus it is recommended to leave **ugrid_coordinates_file** unset if all required elements are found in the data file. If both a data and coordinates file are provided, the shape of the latitude and longitude dimensions are cross-referenced as a limited enforcement measure that the coordinates file and data file belong to the same UGRID.
+
+Required Unstructured Grid Metadata
+===================================
+
+To correctly parse the UGRID topology, MET needs the following information that must be contained within the UGRID coordinates file or the UGRID data file. It is often the case that the variable names for these elements differ between UGRIDs. If the user is not using one of the supported UGRIDs, they will need to define the **ugrid_metadata_map** in a custom configuration file that they provide on the command line using the **-ugrid_config** command line argument, which tells MET the variable names the correspond to the following elements:
+
+Dimension name for UGRID cells (required)
+UGRID convention element name: dim_face
+Coordinate variable name for the latitude of each UGRID cell (required)
+UGRID convention element name: lat_face
+Coordinate variable name for the longitude of each UGRID cell (required)
+UGRID convention element name: lon_face
+Dimension name for the time coordinate (required)
+UGRID convention element name: dim_time
+Coordinate variable name for the time coordinate (required)
+UGRID convention element name: time
+Dimension name for the vertical coordinate, if present (optional)
+UGRID convention element name: dim_vert
+Coordinate variable name for the vertical coordinate, if present (optional)
+UGRID convention element name: vert_face
+
+Unstructured Grid Configuration Elements
+========================================
+
+In the PointStat and GridStat config files, a user can control aspects of the UGRID capability.
+
+**ugrid_dataset [optional]**
+If the UGRID dataset is supported by MET, then this item is set to the string identifying the UGRID. Currently supported ugrid_dataset options include “lfric”, and “mpas”. If this item is not set, the user must provide a separate UGRID configuration file on the command line using the **-ugrid_config** command line argument.
+
+**ugrid_max_distance_km [default: 0 km]**
+For PointStat, this is the distance from each UGRID cell center that PointStat will search outward to collect observations to use for verification. The default is 0 km, which by default will use the closest observation to the UGRID cell for verification. For GridStat, this is the distance from each UGRID forecast cell center to search for observation grid cells to include when interpolating to the UGRID forecast cell locations. Currently, only the nearest point observation or gridded observation cell is used.
+
+**ugrid_coordinates_file [optional]**
+The absolute path to the ancillary NetCDF file that describes the UGRID topology. This file is checked for the required UGRID metadata first, and the data file is checked second and takes precedence over this file if the same elements are found in both files.
+
+**ugrid_metadata_map [required if not using a supported UGRID]**
+The mapping dictionary which allows a user to specify the variable names of the required UGRID topology elements that are required by MET. For "lfric" and "mpas", **ugrid_metadata_map** comes pre-configured with MET and a user can simply set **ugrid_dataset**. Otherwise, the user must create a separate UGRID configuration file containing **ugrid_metadata_map** and provide it on the command line using the **-ugrid_config** command line argument.
+
+Unstructured Grid Limitations
+=============================
+
+We anticipate expanding the UGRID capabilities in MET in the near future. Until then, users are encouraged to be mindful of the following limitations:
+
+1. In GridStat, there is no support for verifying directly on a UGRID. The UGRID (either forecast, obs, or both) must be regridded to a structured verification grid prior to verification being performed. It is recommended that the user define a custom verification grid tailored to their UGRID characteristics, being mindful that the regridding can be slow for large verification grids.
+
+2. Data at cell edges and nodes are currently not supported, only those variables which have data at the cell centers are supported. Users should note in particular that wind components that are typically derived using data at cell edges are currently unsupported.
+
+3. No aggregation methods of point observations within the **ugrid_max_distance_km** are supported except NEAREST, and no aggregation methods of gridded observations within the **ugrid_max_distance_km** are supported except NEAREST.

docs/Users_Guide/config_options.rst

@@ -1047,6 +1047,8 @@ to be verified. This dictionary may include the following entries:
                                   single model level.
        file_type = NETCDF_NCCF;   NetCDF following the Climate Forecast
                                   (CF) convention.
+       file_type = NETCDF_UGRID;  NetCDF containing data on an
+                                  unstructured grid.
        file_type = PYTHON_NUMPY;  Run a Python script to load data into
                                   a NumPy array.
        file_type = PYTHON_XARRAY; Run a Python script to load data into

docs/Users_Guide/grid-stat.rst

@@ -172,6 +172,7 @@ The usage statement for the Grid-Stat tool is listed below:
          fcst_file
          obs_file
          config_file
+         [-ugrid_config config_file]
          [-outdir path]
          [-log file]
          [-v level]
@@ -191,13 +192,15 @@ Required Arguments for grid_stat
 Optional Arguments for grid_stat
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-4. The **-outdir path** indicates the directory where output files should be written.
+4. The **-ugrid_config** option provides a way for a user to provide a separate config file with metadata about their UGRID.
 
-5. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file.
+5. The **-outdir path** indicates the directory where output files should be written.
 
-6. The **-v level** option indicates the desired level of verbosity. The contents of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity above 1 will increase the amount of logging.
+6. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file.
 
-7. The **-compress level** option indicates the desired level of compression (deflate level) for NetCDF variables. The valid level is between 0 and 9. The value of "level" will override the default setting of 0 from the configuration file or the environment variable MET_NC_COMPRESS. Setting the compression level to 0 will make no compression for the NetCDF output. Lower number is for fast compression and higher number is for better compression.
+7. The **-v level** option indicates the desired level of verbosity. The contents of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity above 1 will increase the amount of logging.
+
+8. The **-compress level** option indicates the desired level of compression (deflate level) for NetCDF variables. The valid level is between 0 and 9. The value of "level" will override the default setting of 0 from the configuration file or the environment variable MET_NC_COMPRESS. Setting the compression level to 0 will make no compression for the NetCDF output. Lower number is for fast compression and higher number is for better compression.
 
 An example of the grid_stat calling sequence is listed below:
 

docs/Users_Guide/point-stat.rst

@@ -277,6 +277,7 @@ The usage statement for the Point-Stat tool is shown below:
          fcst_file
          obs_file
          config_file
+         [-ugrid_config config_file]
          [-point_obs file]
          [-obs_valid_beg time]
          [-obs_valid_end time]
@@ -298,17 +299,19 @@ Required Arguments for point_stat
 Optional Arguments for point_stat
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-4. The **-point_obs** file may be used to pass additional NetCDF point observation files to be used in the verification. Python embedding for point observations is also supported, as described in :numref:`pyembed-point-obs-data`.
+5. The **-ugrid_config** option provides a way for a user to provide a separate config file with metadata about their UGRID.
 
-5. The **-obs_valid_beg** time option in YYYYMMDD[_HH[MMSS]] format sets the beginning of the observation matching time window, overriding the configuration file setting.
+5. The **-point_obs** file may be used to pass additional NetCDF point observation files to be used in the verification. Python embedding for point observations is also supported, as described in :numref:`pyembed-point-obs-data`.
 
-6. The **-obs_valid_end** time option in YYYYMMDD[_HH[MMSS]] format sets the end of the observation matching time window, overriding the configuration file setting.
+6. The **-obs_valid_beg** time option in YYYYMMDD[_HH[MMSS]] format sets the beginning of the observation matching time window, overriding the configuration file setting.
 
-7. The **-outdir path** indicates the directory where output files should be written. 
+7. The **-obs_valid_end** time option in YYYYMMDD[_HH[MMSS]] format sets the end of the observation matching time window, overriding the configuration file setting.
 
-8. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. 
+8. The **-outdir path** indicates the directory where output files should be written. 
 
-9. The **-v level** option indicates the desired level of verbosity. The value of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity will increase the amount of logging.
+9. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. 
+
+10. The **-v level** option indicates the desired level of verbosity. The value of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity will increase the amount of logging.
 
 An example of the point_stat calling sequence is shown below: