Feature 1546 unit_tests

[JohnHalleyGotway]

Expected Differences

Collaborated with @georgemccabe on issue #1546. We would like to split up MET's existing unit tests into multiple groups that can run concurrently. This will enable the tests run much faster via GitHub actions than running serially. However, the dependencies between the unit test xml files are not consistently documented. This PR does two things:

1. Add a consist "Requires" comment to the top of each unit test XML file to list any XML pre-requisites.
2. Modify the output directory for a handful of tests so that unit_{name}.xml writes all of its output to a {name} output directory. Following this convention makes the regression test diffing logic much simpler.

• Do these changes introduce new tools, command line arguments, or configuration file options? [No]
  
  If yes, please describe:
  

• Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]
  
  If yes, please describe:
  

Pull Request Testing

• Describe testing already performed for these changes:
  

1. Ran the following commands from the test/xml directory to confirm that the documented pre-reqs are accurate.

for file in `ls -1 *.xml`; do
  short=`echo $file | sed -r 's/unit_//g' | sed -r 's/\.xml//g'`
  echo ========== $file ========= >> output.txt
  grep Requires $file >> output.txt
  grep OUTPUT_DIR $file | egrep -v "&OUTPUT_DIR;/$short" | egrep -v ENTITY >> output.txt
done

I reviewed output.txt and compared the documented requirements with the OUTPUT_DIR strings that show up. Note that unit_met_test_scripts.xml can be ignored since it defines OUTPUT_DIR = $MET_TEST_OUTPUT/met_test_scripts already.

2. I ran detailed dependency testing in kiowa:/d1/projects/MET/MET_pull_requests/met-10.1.0/met-10.1.0_beta6/feature_1546/dependency_testing. In the run_dependency_test.sh script, for each XML file, it runs any pre-reqs for it followed by that XML and writes output to an independent output/{xml_file_name} directory. If all these tests run without error, then the defined pre-reqs are accurate.

3. I ran a full regression test vs develop in kiowa:/d1/projects/MET/MET_pull_requests/met-10.1.0/met-10.1.0_beta6/feature_1546. Please see an explanation of the diffs further down in this PR.

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  
  No additional testing needed, but please review the tests I already performed.

• Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No]
  None needed.

• Do these changes include sufficient testing updates? [Yes]
  These are all testing updates.

• Will this PR result in changes to the test suite? [Yes]
  
  If yes, describe the new output and/or changes to the existing output:
  

Many differences are flagged in kiowa:/d1/projects/MET/MET_pull_requests/met-10.1.0/met-10.1.0_beta6/feature_1546/comp_dir.log. They are explained below.

1. The OBS_ERROR string is changed to OBSERR in 'test_output/stat_analysis/AGG_STAT_ORANK_ECNT_dump.stat', as expected.

2. 58 files are moved from one output directory to another and/or renamed. I tested by using the old directories/names to defines links to the new output files. Then I re-ran the comp_dir.sh script to check for diffs. All remaining differences are expected:

• Renaming OBS_ERROR to OBSERR

• Changing OBS:2015080700_bad_record.nc to OBS_2015080700_bad_record.nc

• Please complete this pull request review by [Monday 1/24/22].
  

Pull Request Checklist

See the METplus Workflow for details.

• Complete the PR definition above.
• Ensure the PR title matches the feature or bugfix branch name.
• Define the PR metadata, as permissions allow.
  Select: Reviewer(s)
  Select: Organization level software support Project or Repository level development cycle Project
  Select: Milestone as the version that will include these changes
• After submitting the PR, select Linked issues with the original issue number.
• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• Close the linked issue and delete your feature or bugfix branch from GitHub.

[georgemccabe]

These changes look good. It is useful to have the list of required test files that need to run before a given test inside each test file. Ensuring that each unit test file writes all of its output to a directory of the same name will make it easier to split up the diff tests in automation. I see there are a few other minor changes to clean things up. I approve.