Feature #2682 default_model

[JohnHalleyGotway]

Expected Differences

• Do these changes introduce new tools, command line arguments, or configuration file options? [No]
  
  If yes, please describe:
  

• Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]
  
  If yes, please describe:
  

Pull Request Testing

• Describe testing already performed for these changes:
  

I carefully inspected the differences flagged by this GHA run for this Pull Request.

It flagged differences in 517 output files.

cd /Users/johnhg/Downloads/diff
find ./ -name "*_OUTPUT*" | wc -l
517

Of the 517, 460 are ASCII. I patched the new output make it look like the old output, replacing FCST with WRF. This proved to me that all there are no unexpected diffs. They're all due to changing WRF to FCST.

find ./ -name "*_OUTPUT*" | egrep -v "\.nc" > output_files

for file in `cat output_files`; do newfile=`echo $file | sed 's/_OUTPUT/_OUTPUT2/g'`; cat $file | sed -r 's/V12.0.0 +FCST/V12.0.0 WRF/g' | sed 's/(FCST)/(WRF)/g' > ${newfile}; done

for file in `cat output_files`; do echo $file; truthfile=`echo $file | sed 's/OUTPUT/TRUTH/g'`; newfile=`echo $file | sed 's/OUTPUT/OUTPUT2/g'`; diff -w $truthfile $newfile; done

For the 57 NetCDF files, all diffs appear as follows in comp_dir.log:

ERROR: NetCDF headers differ:
96c96
< 		:model = "WRF" ;
---
> 		:model = "FCST" ;

Note that an expected TC-Diag diff is also present due to PR #2680.

COMPARING tc_diag/tc_diag_AL092022_GFSO_2022092400_cyl_grid_parent.nc
file1: /data/output/met_test_truth/tc_diag/tc_diag_AL092022_GFSO_2022092400_cyl_grid_parent.nc
file2: /data/output/met_test_output/tc_diag/tc_diag_AL092022_GFSO_2022092400_cyl_grid_parent.nc
ERROR: NetCDF headers differ:
3,4c3,4
<       track_line = 5 ;
<       time = 5 ;
---      
>       track_line = 49 ;
>       time = 22 ;

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  
  Review config file changes, and double-check the file-comparison logic I described above to make sure you agree with the approach.

• Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
  None needed.

• Do these changes include sufficient testing updates? [Yes]
  None needed.

• Will this PR result in changes to the test suite? [Yes]
  
  If yes, describe the new output and/or changes to the existing output:
  
  Differences in 517 output files, replacing the "WRF" model name with "FCST".

• Please complete this pull request review by [Friday 9/15/23].
  

Pull Request Checklist

See the METplus Workflow for details.

• Review the source issue metadata (required labels, projects, and milestone).
• Complete the PR definition above.
• Ensure the PR title matches the feature or bugfix branch name.
• Define the PR metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the version that will include these changes
  Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
• After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• Close the linked issue and delete your feature or bugfix branch from GitHub.

[georgemccabe]

The changes are very straight forward, simply changing WRF to FCST in docs and config files. The logic used to justify the diffs make sense to me.

I just fixed the column naming bug for stat files in the METplus diff logic included in PR dtcenter/METplus#2347. MET still uses its own diff logic that appears to also have a bug naming the columns, but this bug will be fixed when we switch to using the same diff logic for both MET and METplus tests.

The automated tests are rerunning due to the doc changes, but I used a previous run to review the diff output. The changes to the docs shouldn't affect those results.

[JohnHalleyGotway]

I investigated the column offset problem in the MET's diff logic. I ran comp_stat.sh on a single file to replicate the behavior:

bin/comp_stat.sh /Users/johnhg/Downloads/diff/met_test_scripts/point_stat/point_stat_360000L_20070331_120000V_ctc_TRUTH.txt /Users/johnhg/Downloads/diff/met_test_scripts/point_stat/point_stat_360000L_20070331_120000V_ctc_OUTPUT.txt
stat1: /Users/johnhg/Downloads/diff/met_test_scripts/point_stat/point_stat_360000L_20070331_120000V_ctc_TRUTH.txt
stat2: /Users/johnhg/Downloads/diff/met_test_scripts/point_stat/point_stat_360000L_20070331_120000V_ctc_OUTPUT.txt
ERROR: header information mismatch in column DESC

The problem is in that last line should flag the MODEL column rather than DESC.

This problem is fixed by this commit. It was a simple error in the order the list of the header column names. Retesting with this update, I do see the expected error message:

ERROR: header information mismatch in column MODEL