Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensemble-Stat runtime error when obs_thresh <= CDP10 results in 0 pairs being used. #1397

Closed
15 of 18 tasks
JohnHalleyGotway opened this issue Jun 29, 2020 · 0 comments
Closed
15 of 18 tasks
Assignees
Labels
priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center type: enhancement Improve something that it is currently doing
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Jun 29, 2020

Describe the Enhancement

Binbin Zhou from NOAA/EMC pointed out a runtime error he is seeing from the Ensemble-Stat tool. He is applying multiple obs_thresh filtering options, as shown below:

obs_thresh = [ <=CDP10, >CDP10&&<=CDP20, >CDP20&&<=CDP30, >CDP30&&<=CDP40, >CDP40&&<=CDP50, >CDP50&&<=CDP60, >CDP60&&<=CDP70, >CDP70&&<=CDP80, >CDP80&&<=CDP90, >CDP90 ];

However, Ensemble-Stat exits with the following error message:

DEBUG 2: Processing gridded verification HGT/P500 versus HGT/P500, for observation type ANALYS, over region TRP, for interpolation method NEAREST(1), using 2448 matched pairs.
DEBUG 3: Using 0 of 2448 ensemble pairs for observation filtering threshold <=CDP10.
ERROR  :
ERROR  : RPSInfo::set(const PairDataEnsemble &) -> no thresholds provided to compute the RPS line type! Specify thresholds using the "prob_cat_thresh" configuration file option.
ERROR  :

Since 0 pairs survive the obs_thresh filtering of <=CDP10, I would guess that Ensemble-Stat just isn't handling this degenerate case correctly. Need to replicate this behavior, add more checking, and prevent this error condition from occurring so that Ensemble-Stat can run to completion.

Time Estimate

2 days

Sub-Issues

Consider breaking the enhancement down into sub-issues.
No sub-issues required.

Relevant Deadlines

None

Funding Source

None

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required: John Halley Gotway
  • Select scientist(s) or no scientist required: John Optaz

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones
  • Select milestone

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing component: application code requestor: NOAA/EMC NOAA Environmental Modeling Center priority: blocker Blocker labels Jun 29, 2020
@JohnHalleyGotway JohnHalleyGotway added this to the MET 9.1 milestone Jun 29, 2020
@JohnHalleyGotway JohnHalleyGotway added this to To do in MET-9.1-beta2 (7/14/20) via automation Jun 29, 2020
JohnHalleyGotway added a commit that referenced this issue Jun 29, 2020
… pairs down using the obs_thresh filtering criteria can produced 0 matched pairs. This can lead to a runtime error depending on your configuration. Instead, check for 0 pairs after subsetting, and continue to the next processing step.
@JohnHalleyGotway JohnHalleyGotway moved this from To do to Pull request review in MET-9.1-beta2 (7/14/20) Jun 29, 2020
MET-9.1-beta2 (7/14/20) automation moved this from Pull request review to Done Jul 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center type: enhancement Improve something that it is currently doing
Projects
No open projects
Development

No branches or pull requests

2 participants