Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to configure HiRA options in Point-Stat. #1400

Closed
18 tasks done
JohnHalleyGotway opened this issue Jul 2, 2020 · 0 comments
Closed
18 tasks done

Make it easier to configure HiRA options in Point-Stat. #1400

JohnHalleyGotway opened this issue Jul 2, 2020 · 0 comments
Assignees
Labels
MET: Library Code MET: Probability Verifcation priority: medium Medium Priority type: enhancement Improve something that it is currently doing
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Jul 2, 2020

Describe the Enhancement

This enhancement request arose from a met-help question from John Raby:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790

He's trying out the HiRA methodology in Point-Stat and found the prob_cat_thresh settings confusing. He configured Point-Stat to verify multiple variables, including TMP, RH, and winds. He defined cat_thresh for each and requested categorical counts and statistics.

The prob_cat_thresh setting is used in the computation of the Ranked Probability Score (RPS) line type. He assumed those same cat_thresh thresholds would be applied when computing RPS for HiRA, but they are not. Instead, the thresholds are defined by hira.prob_cat_thresh. If hira.prob_cat_thresh is left empty AND climo bins are provided, then Point-Stat will use those thresholds instead. But if none of those are the case, then point-stat will just skip the computation of RPS.

If HiRA RPS output is desired when not using climatologies, then setting the prob_cat_thresh thresholds well becomes very cumbersome. The user would need to define prob_cat_thresh thresholds separately for each field... meaning they'd need to re-define the HiRA dictionary settings for each variable! And that's very cumbersome.

To make this easier, tweak the logic as follows. When computing the HiRA RPS scores, if prob_cat_thresh is empty AND there's no climo bins, simply use the observation cat_thresh thresholds to define RPS. This change will enable users to define the hira dictionary only once and apply it to multiple variables.

Time Estimate

4 hours

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • No sub-issues required.

Relevant Deadlines

None

Funding Source

None

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required: Did not assign a scientist but did ask for Tara to do the pull request review.

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones
  • Select milestone

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing component: library code priority: medium Medium Priority labels Jul 2, 2020
@JohnHalleyGotway JohnHalleyGotway added this to the MET 9.1 milestone Jul 2, 2020
@JohnHalleyGotway JohnHalleyGotway added this to To do in MET-9.1-beta2 (7/14/20) via automation Jul 2, 2020
JohnHalleyGotway added a commit that referenced this issue Jul 4, 2020
JohnHalleyGotway added a commit that referenced this issue Jul 4, 2020
…n the output_flag to write fewer files. Also, add a new test for computing RPS with and without the prob_cat_thresh threshold set.
@JohnHalleyGotway JohnHalleyGotway moved this from To do to Pull request review in MET-9.1-beta2 (7/14/20) Jul 4, 2020
@JohnHalleyGotway JohnHalleyGotway self-assigned this Jul 4, 2020
JohnHalleyGotway added a commit that referenced this issue Jul 7, 2020
* Simple change, replacing calls to n_elements() with calls to n() to make the code more concise.

* Per #1400, reset hira.prob_cat_thresh to obs.cat_thresh if not otherwise set and no climo data is provided.

* Per #1400, the output rps file should be listed as an expected output.

* Per #1400, modify the unit tests for HiRA. Switch from BOTH to STAT in the output_flag to write fewer files. Also, add a new test for computing RPS with and without the prob_cat_thresh threshold set.
@JohnHalleyGotway JohnHalleyGotway moved this from Pull request review to Done in MET-9.1-beta2 (7/14/20) Jul 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: Library Code MET: Probability Verifcation priority: medium Medium Priority type: enhancement Improve something that it is currently doing
Projects
No open projects
Development

No branches or pull requests

2 participants