Add metplus support at nci site

[Scott Wales (ScottWales)]

Add METplus support at NCI

• Add local configuration for ascii2nc and point_stat
• Add a new task metplus_prep_fcst that pulls out the model fields required by metplus and puts them in a netcdf file for metplus using a cset recipe
• Add a new task metplus_prep_obs with a local config that reads our odb2 observation files and puts them in the right format for ascii2nc

Some new features in the nci metplus configs

• The metplus configs have been set up to output logs to the Cylc log directory so they're easily accessible from the Cylc interface
• point_stat has been set up to pull out values by timestamp rather than by index - the sample data I was using had the model output start a few hours after the cycle time so point_stat was matching up the wrong times with observations.

Fixes #2064

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

• Documentation has been updated to reflect change.
• New code has tests, and affected old tests have been updated.
• All tests and CI checks pass.
• Ensured the pull request title is descriptive.
• Ensure rose-suite.conf.example has been updated if new diagnostic added.
• Conda lock files have been updated if dependencies have changed.
• Attributed any Generative AI, such as GitHub Copilot, used in this PR.
• Marked the PR as ready to review.

[Scott Wales (ScottWales)]

A sample config file that shows metplus running at NCI:

[template variables]
SITE="nci-gadi"
CSET_MODEL_COUNT=1
SKIP_WRITE=True

ANALYSIS_LENGTH="PT48H"

METPLUS_GRID_STAT=False
METPLUS_POINT_STAT=True

METPLUS_ANA_DIR="/dev/null"
METPLUS_AREA_DIR="/dev/null"
METPLUS_STN_DIR="/dev/null"

# Where we'll get observations from for metplus
METPLUS_OBS_SYSTEM="access_c3_dn"

CSET_CASE_DATES=[
               ="20240528T0000",
               =]

SPATIAL_SURFACE_FIELD=True
HISTOGRAM_SURFACE_FIELD=True
TIMESERIES_SURFACE_FIELD=True

SURFACE_FIELDS=[
              ='surface_temperature',
              =]

# Cut out variable resolution area for point_stat
SELECT_SUBAREA=True
SUBAREA_EXTENT=75,75,75,75
SUBAREA_TYPE="gridcells"

m1_analysis_offset="PT0H"
m1_data_path="/scratch/dp9/slc548/cylc-run/u-dq090_dne/share/cycle/%Y%m%dT%H00Z/ACCESS-DNE/2p2km/ral3p2/um/em0/umnsaa_pver?%N"
m1_data_period="PT1H"
m1_data_source="filesystem"
m1_date_type="initiation"
m1_name="Darwin CTL"

Output looks like (share/cycle/20240528T0000Z/Point_Stat/point_stat_013000L_20240528_043000V.stat):

VERSION MODEL DESC    FCST_LEAD FCST_VALID_BEG  FCST_VALID_END  OBS_LEAD OBS_VALID_BEG   OBS_VALID_END   FCST_VAR              FCST_UNITS FCST_LEV            OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE
V12.2.1 UM    surface 013000    20240528_043000 20240528_043000 000000   20240528_040000 20240528_050000 air_temperature       K          20240528_043000,*,* t2m     NA        P0      ADPSFC FULL    NEAREST     1           NA          NA         NA         0.05  CNT       65 304.26803 303.83316 304.7029  NA NA  1.78883  1.52546  2.16312 NA NA 304.8583  304.39012 305.32649 NA NA  1.92588  1.64233  2.32883 NA NA 0.84776 0.76127 0.90462 NA NA NA NA 0 0 0 -0.59027 -0.84148 -0.33907 NA NA 1.03331 0.88117 1.24951 NA NA 0.99806 NA NA 0.94789 NA NA  1.39972 NA NA  1.0513 NA NA 1.1831  NA NA -2.00475 NA NA -1.3479  NA NA -0.59799 NA NA 0.021576 NA NA 0.8402  NA NA 1.36948 NA NA 0.67081 NA NA NA NA NA NA NA 0.34842 NA NA 0.62262 NA NA NA NA NA NA NA NA NA NA NA 0.0038808 NA NA
V12.2.1 UM    surface 013000    20240528_043000 20240528_043000 000000   20240528_040000 20240528_050000 relative_humidity     %          20240528_043000,*,* rh2m    NA        P0      ADPSFC FULL    NEAREST     1           NA          NA         NA         0.05  CNT       67  50.28918  46.9893   53.58906 NA NA 13.78122 11.77853 16.61182 NA NA  52.08626  48.96266  55.20985 NA NA 13.04501 11.1493  15.72439 NA NA 0.90796 0.85401 0.94259 NA NA NA NA 0 0 0 -1.79708 -3.18577 -0.40839 NA NA 5.79954 4.95675 6.99074 NA NA 0.9655  NA NA 4.72589 NA NA 36.3622  NA NA 33.1327 NA NA 6.03011 NA NA -9.65103 NA NA -4.73282 NA NA -2.04311 NA NA 1.5435   NA NA 5.35937 NA NA 6.27632 NA NA 3.0753  NA NA NA NA NA NA NA 3.22949 NA NA 0.78632 NA NA NA NA NA NA NA NA NA NA NA 0.11577   NA NA

[github-actions]

[Vinod Kumar (vinodk-bom)]

I have tested the workflow at 7ac8153 and happy for the changes to be merged when you are ready.

[JorgeBornemann (JorgeBornemann)]

Thanks a lot, and sorry for the delay in reviewing, with the new tasks I had to think carefully about the implications for portability across sites, which is nicely covered in the PR, so no worries there.
There will be some more work required to converge on file naming, etc..., but, as we don't know yet the requirements across all partners and for more obs and stats types, we will deal with it later.