Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #1825 #1829

Merged
merged 185 commits into from
Jun 13, 2021
Merged

Update develop-ref after #1825 #1829

merged 185 commits into from
Jun 13, 2021

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Jun 13, 2021

Pull Request Testing

Update develop-ref after PR #1825 for issue #1749. This PR makes 3 changes that affect the output of the NB:

  • Updates the version number from v10.0.0 to v10.1.0, which triggers differences in all the MODE and MTD output files.
  • Adds 4 columns to the MCTS line type (HSS_EC, HSS_EC_BCL, HSS_EC_BCU, EC_VALUE)
  • Adds 1 column to the MCTC line type (EC_VALUE)

All of the resulting differences in the NB20210613 output are expected.

  • Describe testing already performed for these changes:

    The NB for develop failed on 20210613 with these changes. And it changed 35 output files:
egrep -i "file1|ERROR:" test_regression_20210613.log | egrep -B 1 "ERROR:" | grep file1 | wc -l
35

The breakdown is: 17 mtd files, 16 from grid_stat, and 2 from stat_analysis. I inspected the diffs and see they are all as expected.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    No additional testing needed.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
    This PR serves as documentation of the change.

  • Do these changes include sufficient testing updates? [Yes]

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    As described above.

  • Please complete this pull request review by [06/13/21].

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s), Project(s), and Milestone
  • After submitting the PR, select Linked Issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

MET Tools Test Account and others added 24 commits May 14, 2021 10:49
* Per #1798, fix up the read_tmpe_dataplane.py script to handle a grid string or dictionary.

* Per #1798, add a test to unit_python.xml to exercise this bugfix.
…pport extra-tropical cyclone tracks not verified against BEST tracks.
* Updated compile script and added assocaited config files

* Added jet config file

* Updated orion file

* Added new stampede config file and modulefiles for various machines
…rt plotting of extra-tropical cyclone tracks that aren't verified against BEST tracks.
…branch so that they'll be available for future releases.
Adding GitHub Discussions information
* #1581 Initial release

* #1581 Added met_nc_point_obs.cc met_nc_point_obs.h

* Removed nc_ in function names and moved them to the struct members

* #1581 Added HDR_TYPE_ARR_LEN

* #1581 Changed API calls (API names)

* #1581 Cleanup

* #1581 Removed duplicated definitions:hdr_arr_len, hdr_typ_arr_len, and obs_arr_len

* #1581 Removed duplicated definitions:hdr_arr_len and obs_arr_len

* #1581 Removed duplicated definitions: str_len, hdr_arr_len, and obs_arr_len

* Added vx_nc_obs library

* #1581 Using common APIs

* #1581 Corrected API calls because of renaming for common APIs

* #1581 Moved function from nc_obs_util to nc_point_obs2

* #1581renamed met_nc_point_obs to nc_point_obs

* #1581 API ica changed from obs_vars to nc_point_obs

* #1581 Initial release

* #1581 Renamed from met_nc_point_obs to nc_point_obs

* 1581 Renamed met_nc_point_obs to nc_point_obs

* Per #1581, update the Makefile.am for lidar2nc to fix a linker error. Unfortunatley, the vx_config library now depends on the vx_gsl_prob library. threshold.cc in vx_config includes a call to normal_cdf_inv(double, double, double) which is defined in vx_gsl_prob. This adds to the complexity of dependencies for MET's libraries. Just linking to -lvx_gsl_prob one more time does fix the linker problem but doesn't solve the messy dependencies.

* #1581 Added method for NcDataBuffer

* Cleanup

* #1581 Cleanup

* #1581 Cleanup

* #1591 Cleanup

* #1591 Corrected API

* #1581 Avoid reading PB header twice

* #1581 Warning if PB header is not defined but read_pb_hdr_data is called

* #1581 Cleanup libraries

* 1581 cleanup

* 1581 cleanup

* 1581 cleanup

* #1581 Cleanup for Fortify (removed unused variables)

* #1581 Cleanup

* #1581 Cleanup

* #1581 Use MetNcPointObsIn instead of MetNcPointObs

* #1581 Use MetNcPointObsOut instead of MetNcPointObs2Write

* #1581 Separated nc_point_obs2.cc to nc_point_obs_in.cc and nc_point_obs_out.cc

* #1581 Renamed nc_point_obs2.cc to nc_point_obs_in.cc And added add nc_point_obs_in.h nc_point_obs_out.h nc_point_obs_out.cc

* #1581 Removed APIs related with writing point obs

* #1581 Changed copyright years

* #1581 Cleanup

* #1581 Updated copyright year

* #1581 Cleanup

* #1581 Reanmed read_obs_data_strings to read_obs_data_table_lookups

* #1581 Reanmed read_obs_data_strings to read_obs_data_table_lookups

* #1581 Added more APIs

Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu>
* Per issue #1792, Change -type from optional to required. Set default_mask_type to MaskType_None. Added a check on mask_type to see if it's set and print error message accordingly.

* Update test_gen_vx_mask.sh

* For the first test, added -type poly, since the masking type is now required. SL

* For all of the Poly unit tests added -type poly to the command line. The mask type is now required. SL

* Modified document to indicate that -type string (masking type) is now required on the command line for gen_vx_mask. SL

* Update met/docs/Users_Guide/masking.rst

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update met/src/tools/other/gen_vx_mask/gen_vx_mask.cc

Co-authored-by: johnhg <johnhg@ucar.edu>

Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
…05. We changed -type from optional to required, but missed adding the -type option in unit_met_test_scripts.xml and unit_ref_config.xml. This is a hotfix to resolve that.
* testing new anchoring link idea.

* testing without the bold asterik

* tinkering with the look

* another attempt

* another attempt #2

* another attempt #3

* another attempt #4

* making sure new anchor works as expected.

* seeing if link will save with spaces instead of dashes

* need underscores to link

* is it fixed?

* testing

* testing 2

* testing 4

* testing 5

* testing 6

* testing 7

* testing 8

* going back to test original problem

* able to link with spaces instead of underscores. Testing if a return is possible to keep under 79 character limit.

* double checking everything is still working.

* DO NOT break ref lines apart, it won't work.

* trying a shorter name.

* continuing to add anchors

* updating lines 1946 thru 2214 with anchors

* updating lines 2214 thru 3371 with anchors

* updating lines 3371 to the end with anchors

* testing anchor

* testing anchor

* testing anchor 3

* testing anchor 4

* testing anchor 45 percent

* testing anchor final half

* fixing typo

* numbering fcst, obs_1 and 2 to create different links.

* finding more anchors that need numbers to keep them separate.

* fixing warnings

* fixing warnings

* fixing typo
* Per #1749, updating the MET version number from 10.0 to 10.1 prior to adding new columns of output to existing line types.

* Per #1749, adding 10.1 columns to the Makefile.am

* Per #1749, changes for the mechanics of adding the HSS_EC statistic to the MCTS line type. Still need to acutally compute it and make the expected correct value configurable.

* Per #1749, add hss_ec_value as a configurable option for Point-Stat and Grid-Stat. Still need to actually compute it correctly, add it to other test config files, add support to series_analysis/stat_analysis, update the docs, and make writeup corresponding issues for other METplus components.

* Per #1749, fix the column offsets for the HSS_EC columns.

* Per #1749, add correct definition of HSS_EC.

* Per #1749, pass hss_ec_value from the config file into the computation of the MCTS statistics.

* Per #1749, add hss_ec_value entry to all the Grid-Stat config files.

* Per #1749, update the documentation about the HSS_EC statistic.

* Per #1749, add the -hss_ec_value job command option to Stat-Analysis.

* Per #1749, no real code changes here. Just changing to consistent ordering with hss_ec_value preceeding rank_corr_flag.

* Per #1749, update docs for stat_analysis supporting hss_ec_value.

* Per #1749, add HSS_EC to Series-Analysis, but only with a constant hss_ec_value for now.

* Per #1749, add EC_VALUE to the MCTC line type definition.

* Per #1749, move ECvalue from the MCTSInfo class into the ContingencyTable class so that it's available to be included in the MCTC output line type.

* Per #1749, update point_stat, grid_stat, and series_analysis to accomodate the move of ECvalue from the MCTSInfo class to the ContingencyTable class.

* Per #1749, update library code to write EC_VALUE to the MCTC line type and update the User's Guide docs.

* Per #1749, update stat_analysis code for the addition of EC_VALUE in the MCTC line type.

* Per #1749, write EC_VALUE to the MCTC output line type.

* Per #1749, store the ec_value that was actually used to compute the stats.

* Per #1749, parsing EC_VALUE from the MCTC line type.

* Per #1749, move the MCTC EC_VALUE column to the end of the line, as requested by METdatadb.

* Per #932, need to write MCTS HSS_EC value to temp file during the bootstrapping process.

* Added new reference for Ou 2016

* Layout correction

* Added generalized HSS, removed word from HSS_EC

* Per #1749, change the hss_ec_value config entry to match new conventions.

Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
* Per #1826, add update the version in the docs to 10.1.0-beta1 and add release notes for this development version.

* Per #1826, change the beta1 release date to 6/11 so that I can do it today.
@JohnHalleyGotway JohnHalleyGotway added this to In progress in MET-10.1.0-beta2 (9/01/21) via automation Jun 13, 2021
@JohnHalleyGotway JohnHalleyGotway added this to the MET 10.1.0 milestone Jun 13, 2021
@JohnHalleyGotway JohnHalleyGotway linked an issue Jun 13, 2021 that may be closed by this pull request
20 tasks
@JohnHalleyGotway JohnHalleyGotway merged commit fe14644 into develop-ref Jun 13, 2021
MET-10.1.0-beta2 (9/01/21) automation moved this from In progress to Done Jun 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

HSS enhancement, configurable option for statistical calculation
10 participants