Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix 1694 main_v9.1 grid_diag #1709

Merged
merged 1 commit into from Mar 10, 2021

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Mar 10, 2021

Pull Request Testing

Here's the old grid_diag output on kiowa:

ncdump /d1/projects/MET/MET_regression/main_v9.1/NB20210310/MET-main_v9.1/test_output/grid_diag/grid_diag_APCP_06_FCST_OBS.nc

1D histogram counts are duplicated!

 hist_FCST_APCP_A06 = 170628, 1427, 1144, 786, 532, 401, 285, 283, 221, 241, 
    192, 181, 180, 139, 112, 98, 87, 66, 77, 84, 70, 49, 34, 42, 23, 21, 10, 
    5, 5, 2, 3, 2, 4, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;

 hist_OBS_APCP_A06 = 170628, 1427, 1144, 786, 532, 401, 285, 283, 221, 241, 
    192, 181, 180, 139, 112, 98, 87, 66, 77, 84, 70, 49, 34, 42, 23, 21, 10, 
    5, 5, 2, 3, 2, 4, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;

Here's the new (correct) grid_diag output on kiowa:

ncdump /d1/projects/MET/MET_pull_requests/met-10.0.0_beta5/bugfix_1694/MET-bugfix_1694_main_v9.1_grid_diag_into_main_v9.1/test_output/grid_diag/grid_diag_APCP_06_FCST_OBS.nc

1D histogram counts are unique and correct:

 hist_FCST_APCP_A06 = 83326, 1160, 905, 608, 412, 306, 220, 226, 186, 181, 
    155, 151, 148, 107, 81, 76, 69, 54, 61, 67, 60, 43, 30, 36, 21, 16, 8, 3, 
    2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;

 hist_OBS_APCP_A06 = 87302, 267, 239, 178, 120, 95, 65, 57, 35, 60, 37, 30, 
    32, 32, 31, 22, 18, 12, 16, 17, 10, 6, 4, 6, 2, 5, 2, 2, 3, 2, 3, 2, 4, 
    0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;
  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    Please compare the modified files to the change for develop PR:
    https://github.com/dtcenter/MET/pull/1696/files

If there's anything you'd like to run, you can find it compiled on kiowa in:
/d1/projects/MET/MET_pull_requests/met-10.0.0_beta5/bugfix_1694/MET-bugfix_1694_main_v9.1_grid_diag_into_main_v9.1/met

  • Do these changes include sufficient documentation and testing updates? [Yes]

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    The output for an existing unit_grid_diag.xml test will change, as it did in the develop branch. So I'll need to update the reference version when it does.

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s), Project(s), and Milestone
  • After submitting the PR, select Linked Issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@JohnHalleyGotway JohnHalleyGotway added this to the MET 9.1.2 (bugfix) milestone Mar 10, 2021
@JohnHalleyGotway JohnHalleyGotway added this to In progress in METplus Version 3.1 Support via automation Mar 10, 2021
Copy link
Collaborator

@jprestop jprestop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I compared the modified files to the change for develop PR. Thanks for the link! The changes were comparable. I did not do any other testing. I approve this request.

@JohnHalleyGotway JohnHalleyGotway moved this from In progress to Done in METplus Version 3.1 Support Mar 10, 2021
@JohnHalleyGotway JohnHalleyGotway merged commit 9e494f6 into main_v9.1 Mar 10, 2021
@JohnHalleyGotway JohnHalleyGotway deleted the bugfix_1694_main_v9.1_grid_diag branch March 10, 2021 22:47
JohnHalleyGotway added a commit that referenced this pull request Mar 12, 2021
* Add debug level 4 message to list out the number of GRIB2 records inventoried. This helps debugging issues with MET potentially not reading all input GRIB2 records on WCOSS.

* Bugfix 1554 main_v9.1 ncdump (#1555)

* Bugfix 1562 main_v9.1 grid_diag (#1563)

* Per #1562, check for bad data values before adding data to the PDF's for grid_diag.

* Per #1562, removing the poly = CONUS.poly mask from GridDiagConfig_TMP. That settting masked a problem in the handling of missing data. Exercising the mask.poly option is tested in another unit test. This will change the output and break the nightly build, but that's good since we'll do more thorough testing.

* Per #1508, change the verbosity in unit_tc_gen.xml from -v 2 to -v 5 to print out some additional log messages that may help in debugging the intermittent file list failure.

* Feature 1572 v9.1.1 (#1573)

* Per #1572, delete the docs/version file as it is not needed here.

* Per #1572, update the version number to 9.1.1.

* Per #1572, list the met-9.1.1 release date as 20201119 for the docs.

* Per #1572, add release notes for the met-9.1.1 verison.

* Per #1572, add release notes for met-9.1.1 version.

* Per #1572, let's try to release met-9.1.1 today 11/18.

* Correct GitHub link.

* Fix small typo in release notes.

* Bugfix 1618 main v91 pb2nc (#1622)

Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>

* Update pull_request_template.md

* Per #1638, apply the same 3 fixes to the main_v9.1 branch to be included in the next bugfix release for met-9.1. (#1640)

* Per #1646, one line fix for cut-and-paste error. (#1648)

* Adding necessary files for ReadTheDocs (#1703)

* Per #1706, update aggr_ecnt_lines() to also add an entry to the skip_ba array of booleans. This is used to keep track of whether or not the point should be excluded from the statistics and is used in the Ensemble-Stat. This same aggregation code in the vx_statistics library is used by Stat-Analysis. Since Stat-Analysis failed to populate that array, it led to an array parsing error in ECNTInfo::set() when trying to compute the sum of the weights. (#1707)

* Per #1694, exactly the same set of bugfix changes but applied to main_v9.1 rather than the develop branch. (#1709)

* Feature 1710 v9.1.2 (#1711)

* Per #1710, update the release notes and version numbers for the 9.1.2 release.

* Per #1710, add a release note about the move to read the docs.

Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
JohnHalleyGotway added a commit that referenced this pull request Mar 20, 2021
* Add debug level 4 message to list out the number of GRIB2 records inventoried. This helps debugging issues with MET potentially not reading all input GRIB2 records on WCOSS.

* Bugfix 1554 main_v9.1 ncdump (#1555)

* Bugfix 1562 main_v9.1 grid_diag (#1563)

* Per #1562, check for bad data values before adding data to the PDF's for grid_diag.

* Per #1562, removing the poly = CONUS.poly mask from GridDiagConfig_TMP. That settting masked a problem in the handling of missing data. Exercising the mask.poly option is tested in another unit test. This will change the output and break the nightly build, but that's good since we'll do more thorough testing.

* Per #1508, change the verbosity in unit_tc_gen.xml from -v 2 to -v 5 to print out some additional log messages that may help in debugging the intermittent file list failure.

* Feature 1572 v9.1.1 (#1573)

* Per #1572, delete the docs/version file as it is not needed here.

* Per #1572, update the version number to 9.1.1.

* Per #1572, list the met-9.1.1 release date as 20201119 for the docs.

* Per #1572, add release notes for the met-9.1.1 verison.

* Per #1572, add release notes for met-9.1.1 version.

* Per #1572, let's try to release met-9.1.1 today 11/18.

* Correct GitHub link.

* Fix small typo in release notes.

* Bugfix 1618 main v91 pb2nc (#1622)

Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>

* Update pull_request_template.md

* Per #1638, apply the same 3 fixes to the main_v9.1 branch to be included in the next bugfix release for met-9.1. (#1640)

* Per #1646, one line fix for cut-and-paste error. (#1648)

* Adding necessary files for ReadTheDocs (#1703)

* Per #1706, update aggr_ecnt_lines() to also add an entry to the skip_ba array of booleans. This is used to keep track of whether or not the point should be excluded from the statistics and is used in the Ensemble-Stat. This same aggregation code in the vx_statistics library is used by Stat-Analysis. Since Stat-Analysis failed to populate that array, it led to an array parsing error in ECNTInfo::set() when trying to compute the sum of the weights. (#1707)

* Per #1694, exactly the same set of bugfix changes but applied to main_v9.1 rather than the develop branch. (#1709)

* Feature 1710 v9.1.2 (#1711)

* Per #1710, update the release notes and version numbers for the 9.1.2 release.

* Per #1710, add a release note about the move to read the docs.

* #1715 Do not combined if there are no overlapping beteewn TQZ and UV records

* #1715 Do not combined if there are no overlapping beteewn TQZ and UV records

* #1715 Do not combined if there are no overlapping beteewn TQZ and UV records

* #1715 Added blank line for Warning

* #1715 Added a blank line for Error

* Bugfix 1716 main_v9.1 perc_thresh (#1721)

* Per #1716, committing the same bugfix to the main_v9.1 branch for inclusion in the 9.1.3 bugfix release.

* Per #1716, change SFP50 example to SFP33.3 to show an example of using floating point percentile values.

* Per #1723, update the verison number to 9.1.3. (#1730)

* Update pull_request_template.md

Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Fix Grid-Diag bug when reading the same variable name from multiple data sources.
2 participants