Feature #3220 met_version_diff

[JohnHalleyGotway]

Pull Request Testing

This PR proposes the following changes to the METplus diff logic:

• Update _nc_fields_are_equal() to remove unneeded reference to var_a[0] in a log message which causes an exception for 1D NetCDF variables.

• Update _handle_file_list_files() logic to also pass the filenames as input. Also consider them to be file lists if the file names contain file_list. For file lists, replace instances of MET-*/ with MET/ to ignore unimportant diffs in the path to MET. Update that function to return the sanitized version of lines_a and lines_b so they can be used as input to the diffing logic below.

• Describe testing already performed for these changes:
  
  Note that the diffs failed for the MET Nightly Build on seneca in /d1/projects/MET/MET_regression/develop/NB20260206.
  On seneca, I reran the diff using the following commands:

cd /d1/personal/johnhg/METplus/development/METplus-7.0.0/beta2/METplus-feature_3220_met_version_diff
ln -sf /d1/projects/MET/MET_regression/develop/NB20260206 .
export METPLUS_DIR=.
NB20260206/MET-develop/internal/test_unit/python/comp_dir.py \
NB20260206/MET-develop-ref/test_output \
NB20260206/MET-develop/test_output >& comp_dir_feature_3220.log

And I confirmed that differences are no longer flagged, even with the version difference where MET-develop-ref has V12.2.0 and MET-develop has V13.0.0.

Number of files compared = 1251
Number of files skipped = 19
SUCCESS: No differences found in any files

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  
  Confirm that all the GHA tests pass.

• Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]

• Do these changes include sufficient testing updates? [Yes or No]

• Will this PR result in changes to the test suite? [Yes or No]
  
  If yes, describe the new output and/or changes to the existing output:
  

• Do these changes introduce new SonarQube findings? [Yes or No]
  
  If yes, please describe:

• Please complete this pull request review by [Fill in date].
  

Pull Request Checklist

See the METplus Workflow for details.

• Add any new Python packages to the METplus Components Python Requirements table.
• For any new datasets, an entry to the METplus Verification Datasets Guide.
• Review the source issue metadata (required labels, projects, and milestone).
• Complete the PR definition above.
• Ensure the PR title matches the feature or bugfix branch name.
• Define the PR metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the version that will include these changes
  Select: Coordinated METplus-X.Y Support project for bugfix releases or METplus-Wrappers-X.Y.Z Development project for official releases
• After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• Close the linked issue and delete your feature or bugfix branch from GitHub.

[coveralls]

Pull Request Test Coverage Report for Build 21832047976

Details

• 8 of 8 (100.0%) changed or added relevant lines in 1 file are covered.
• 79 unchanged lines in 4 files lost coverage.
• Overall coverage increased (+0.008%) to 92.294%

---

Files with Coverage Reduction	New Missed Lines	%
metplus/wrappers/point_stat_wrapper.py	9	89.22%
metplus/wrappers/ensemble_stat_wrapper.py	17	79.51%
metplus/util/met_config.py	24	94.51%
metplus/util/config_metplus.py	29	88.24%

Totals
Change from base Build 21759736578:	0.008%
Covered Lines:	9569
Relevant Lines:	10368

---

💛 - Coveralls

[georgemccabe]

I made a change to your branch. Please review the new changes.

To handle branch names that include numbers, since we typically name branches with the issue number. Previously, the regex substitution only allowed alphabet (a-zA-Z) and dash characters. Now, it finds all characters that aren't a forward slash and an explicit forward slash, so it will only match characters within the single directory name. I also included a forward slash to the beginning to prevent it from matching other paths with MET included.

I also replaced string find() calls to use in -- find returns 0 (False) if the substring starts at the beginning of the string and returns -1 (True) if the substring is not found. Both of these can result in incorrect results.

To test, I created a few text files and ran the diff script from this branch. There are files that start with file_list and files that have file_list in the filename (both at the beginning and in the middle). The test files that have no-diff in the name should not report any diffs and the files that have one-diff in the name should report a difference.

$ for f in a/*; do echo; echo FILENAME: $f; echo; cat $f; echo; echo FILENAME: b${f:1}; echo; cat b${f:1}; done

FILENAME: a/file_list_no-diff.txt

/some/path/MET-develop/some/other/stuff.txt
/some/path/MET-develop/some/other/stuff2.txt
/some/path/MET-develop/some/other/stuff3.txt

FILENAME: b/file_list_no-diff.txt

/some/path/MET-feature_322/some/other/stuff.txt
/some/path/MET-feature_322/some/other/stuff2.txt
/some/path/MET-feature_322/some/other/stuff3.txt

FILENAME: a/no-diff_file_list.txt

/some/path/MET-develop/some/other/stuff.txt
/some/path/MET-develop/some/other/stuff2.txt
/some/path/MET-develop/some/other/stuff3.txt

FILENAME: b/no-diff_file_list.txt

/some/path/MET-feature_322/some/other/stuff.txt
/some/path/MET-feature_322/some/other/stuff2.txt
/some/path/MET-feature_322/some/other/stuff3.txt

FILENAME: a/no-diff.txt

file_list
/some/path/MET-develop/some/other/stuff.txt
/some/path/MET-develop/some/other/stuff2.txt
/some/path/MET-develop/some/other/stuff3.txt

FILENAME: b/no-diff.txt

file_list
/some/path/MET-feature_322/some/other/stuff.txt
/some/path/MET-feature_322/some/other/stuff2.txt
/some/path/MET-feature_322/some/other/stuff3.txt

FILENAME: a/one-diff_file_list.txt

/some/path/MET-develop/some/other/stuff.txt
/some/path/MET-develop/some/other/stuff2.txt
/some/path/MET-develop/some/other/stuff3.txt

FILENAME: b/one-diff_file_list.txt

/some/path/MET-feature_322/some/other/stuff.txt
/some/path/MET-feature_322/some/other/stuff2.txt
/some/path/MET-feature_322/some/other/stuffy3.txt

FILENAME: a/one-diff.txt

file_list
/some/path/MET-develop/some/other/stuff.txt
/some/path/MET-develop/some/other/stuff2.txt
/some/path/MET-develop/some/other/stuffy.txt

FILENAME: b/one-diff.txt

file_list
/some/path/MET-feature_322/some/other/stuff.txt
/some/path/MET-feature_322/some/other/stuff2.txt
/some/path/MET-feature_322/some/other/stuffy3.txt

Diff results before replacing find with in

---

ERROR SUMMARY:

::error::3 files with differences were found
Text diff
A: /Users/mccabe/tmp/diff_test/a/one-diff.txt
B: /Users/mccabe/tmp/diff_test/b/one-diff.txt
ERROR: File differs: /Users/mccabe/tmp/diff_test/b/one-diff.txt
ERROR: Line differs
A: /some/path/MET/some/other/stuffy.txt
B: /some/path/MET/some/other/stuffy3.txt

Text diff
A: /Users/mccabe/tmp/diff_test/a/file_list_no-diff.txt
B: /Users/mccabe/tmp/diff_test/b/file_list_no-diff.txt
ERROR: File differs: /Users/mccabe/tmp/diff_test/b/file_list_no-diff.txt
ERROR: Line differs
A: /some/path/MET-develop/some/other/stuff.txt
B: /some/path/MET-feature_322/some/other/stuff.txt
ERROR: Line differs
A: /some/path/MET-develop/some/other/stuff2.txt
B: /some/path/MET-feature_322/some/other/stuff2.txt
ERROR: Line differs
A: /some/path/MET-develop/some/other/stuff3.txt
B: /some/path/MET-feature_322/some/other/stuff3.txt

Text diff
A: /Users/mccabe/tmp/diff_test/a/one-diff_file_list.txt
B: /Users/mccabe/tmp/diff_test/b/one-diff_file_list.txt
ERROR: File differs: /Users/mccabe/tmp/diff_test/b/one-diff_file_list.txt
ERROR: Line differs
A: /some/path/MET/some/other/stuff3.txt
B: /some/path/MET/some/other/stuffy3.txt

Diff results after replacing find with in

ERROR SUMMARY:

::error::2 files with differences were found
Text diff
A: /Users/mccabe/tmp/diff_test/a/one-diff.txt
B: /Users/mccabe/tmp/diff_test/b/one-diff.txt
ERROR: File differs: /Users/mccabe/tmp/diff_test/b/one-diff.txt
ERROR: Line differs
A: /some/path/MET/some/other/stuffy.txt
B: /some/path/MET/some/other/stuffy3.txt

Text diff
A: /Users/mccabe/tmp/diff_test/a/one-diff_file_list.txt
B: /Users/mccabe/tmp/diff_test/b/one-diff_file_list.txt
ERROR: File differs: /Users/mccabe/tmp/diff_test/b/one-diff_file_list.txt
ERROR: Line differs
A: /some/path/MET/some/other/stuff3.txt
B: /some/path/MET/some/other/stuffy3.txt

[JohnHalleyGotway]

@georgemccabe sounds good. I updated my branch, recomputed the diffs for NB20260206, and confirmed that none are flagged. So that works for me.

Number of files compared = 1251
Number of files skipped = 19
SUCCESS: No differences found in any files

I think it's still you who should submit the approving review. Hopefully once GitHub is healthy again, the GHA testing workflow will succeed again.