Wind RMSE difference

[wuxx66]

Software Version

coordinated release 6.0.0 (METplus 6.0.0 and MET 12.0.2)

Text

I am working on Australian deterministic model ACCESS-G verification against its own analysis and generating WMO reports using METplus results. I compared the reports with those produced by "Verify" (Our old verification package and originally from ECMWF) and found some difference in wind RMSE.

For example, wind RMSE at 500hPa

To investigate, I recomputed wind RMSE using the U and V component RMSE values from the METplus output. This produced much better agreement with the Verify results:

The difference appears to arise from how wind RMSE is computed. In my METplus config, wind RMSE was probably derived from wind speed (a scalar quantity). In contrast, "Verify" computes wind RMSE using the full wind vector (u, v), combining the component errors.
RMSE_wind = sqrt( RMSE(u)^2 + RMSE(v)^2 )

I looked at the WMO documentation, which clearly states that wind RMSE should be computed as RMS vector wind error. Therefore, the "Verify "system is using the correct method for calculating WMO wind RMSE.

Below is part of my config for Wind.
FCST_GRID_STAT_VAR7_NAME = UGRD
FCST_GRID_STAT_VAR7_LEVELS = P500
FCST_GRID_STAT_VAR7_OPTIONS = is_u_wind=TRUE;

FCST_GRID_STAT_VAR8_NAME = VGRD
FCST_GRID_STAT_VAR8_LEVELS = P500
FCST_GRID_STAT_VAR8_OPTIONS = is_v_wind=TRUE;

FCST_GRID_STAT_VAR9_NAME = WIND
FCST_GRID_STAT_VAR9_LEVELS = P500

GRID_STAT_OUTPUT_FLAG_CNT = BOTH
OBS_GRID_STAT_VAR7_NAME = UGRD
OBS_GRID_STAT_VAR7_LEVELS = P500
OBS_GRID_STAT_VAR7_OPTIONS = is_u_wind=TRUE;

OBS_GRID_STAT_VAR8_NAME = VGRD
OBS_GRID_STAT_VAR8_LEVELS = P500
OBS_GRID_STAT_VAR8_OPTIONS = is_v_wind=TRUE;

OBS_GRID_STAT_VAR9_NAME = WIND
OBS_GRID_STAT_VAR9_LEVELS = P500

Could you help me have a look of this issue?

Thank you

Xiaoxi

[CPKalb]

Hi Xiaoxi:

I want to make sure I understand everything correctly. The first graphic uses the RMSE from MET on the variable named WIND, while the second uses RMSE from MET on either UGRD or VGRD. Is that accurate? Also, is the RMSE you are using for these graphics coming from an output CNT line type (continuous statistics)?

If the RMSE on these graphics is coming from CNT (continuous statistics) output, then it is computed using the scalar wind speed rather than the wind vectors. METplus does have the capability to compute statistics using the wind vectors, but those statistics are output by requesting VL1L2 or VCNT (Vector continuous statistics) output line types. We have documentation on the different vector statistics METplus can compute.

To get VCNT output from METplus, you will want to make sure you have the following in your configuration:

1. Both the U and V wind components are variables in the FCST and OBS data, and the is_u_wind and is_v_wind options are set to TRUE. From the configurations you listed above, it looks like this is already set up.
2. The Vector continuous statistics are requested as output by setting either: GRID_STAT_OUTPUT_FLAG_VCNT = STAT or GRID_STAT_OUTPUT_FLAG_VCNT = BOTH depending on whether you want just the .stat files or the .txt files as well.

Are the graphics above using the the RMSE computed from CNT output or is that RMSVE from VCNT output?

Christina

[wuxx66]

Thank you Christina
Yes. The first graphic uses the RMSE from MET on the variable named WIND, but the second uses RMSE from MET on both UGRD and VGRD. The equation is
RMSE_wind = sqrt( RMSE(u)^2 + RMSE(v)^2 )

The RMSE I was using for these graphics coming from an output [CNT line type].

Thank you for explaining these for me. I will add VCNT in the config and compare the results.

[wuxx66]

Hi Christina,
I have added VCNT and VL1L2 in my config. Here is the plot I produced.

There are only 10 day model data in May, but for the available data, the wind vector RMSVE produced by "METplus" is close enough to "Verify"

[CPKalb]

Hi XXiaoxi:

Great! I'm going to mark this as answered then and close the discussion. If you have additional questions, feel free to start up a new discussion.

Christina