Enhance METcalcpy to aggregate statistics added to the ECNT, VL1L2, VAL1L2, and VCNT line types during the MET 12.0.0 beta4 development cycle

[JohnHalleyGotway]

Describe the Enhancement

Issue dtcenter/METdataio#282 updates the schema to load new wind direction statistics from the VL1L2, VAL1L2, and VCNT line types of MET version 12.0.0 and new ignorance score statistics to the ECNT line type. This issue is to update METcalcpy to update the aggregation logic for the VL1L2, VAL1L2, VCNT, and ECNT line types to handle these new columns of data. Please see @JohnHalleyGotway with any questions you may have about the aggregation logic.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
• Add new wind direction verification statistics for RMSE, Bias, and MAE MET#2395
• Enhance METdataio schema to handle changes to the ECNT, VL1L2, VAL1L2, and VCNT line types during the MET 12.0.0 beta4 development cycle METdataio#282
• Enhance METcalcpy to aggregate statistics added to the ECNT, VL1L2, VAL1L2, and VCNT line types during the MET 12.0.0 beta4 development cycle #361
• Enhance METviewer to plot statistics added to the ECNT, VL1L2, VAL1L2, and VCNT line types during the MET 12.0.0 beta4 development cycle METviewer#517

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Add any new Python packages to the METplus Components Python Requirements table.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

1. For the VL1L2 columns:

• DIR_ME, DIR_MAE, DIR_MSE: Aggregate these simply as a weighted average of the input statistics, where the weight is defined by the TOTAL column.

'vl1l2_dir_me': ['dir_me'],
'vl1l2_dir_mae': ['dir_mae'],
'vl1l2_dir_mse': ['dir_mse'],

2. For the VCNT columns:

• DIR_ME, DIR_MAE, DIR_MSE: Aggregate these simply as a weighted average of the input statistics, where the weight is defined by the TOTAL column.

'vcnt_dir_me': ['dir_me'],
'vcnt_dir_mae': ['dir_mae'],
'vcnt_dir_mse': ['dir_mse'],

• DIR_RMSE: Aggregate DIR_MSE, as described above, and then take the square root.

'vcnt_dir_rmse': ['dir_mse'],

3. For the VAL1L2 columns:

• DIRA_ME, DIRA_MAE, DIRA_MSE: Aggregate these simply as a weighted average of the input statistics, where the weight is defined by the TOTAL column.

'val1l2_dira_me': ['dira_me],
'val1l2_dira_mae': ['dira_mae],
'val1l2_dira_mse': ['dira_mse],

4. For the ECNT columns:

• IGN_CONV_OERR, IGN_CORR_OERR: Aggregate these simply as a weighted average of the input statistics, where the weight is defined by the TOTAL column.

In general, when adding aggregation logic to METcaclpy, recommend testing and validating using MET's Stat-Analysis tool to make sure we get consistent results. e.g. run the -job aggregate and/or -job aggregate_stat job types.