Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend climo_cdf logic from Point-Stat and Grid-Stat over to Ensemble-Stat. #1224

Closed
JohnHalleyGotway opened this issue Dec 11, 2019 · 3 comments
Assignees
Labels
MET: Library Code priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center type: enhancement Improve something that it is currently doing
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

This is an extension of the binned climatology logic for Ensemble-Stat. This issue is based on this response to Binbin about ensemble verification:

I see that Ensemble-Stat is erroring out when trying to use CDP thresholds. I pulled your sample data from Hera down to my machine and was able to replicate the behavior you describe:
ERROR : Simple_Node::check(double, double, double) const -> climatological distribution percentile threshold "<=CDP10" requested with invalid mean (-9999) or standard deviation (-9999).

Here's the short answer... edit your config file by renaming "cat_thresh" to "obs_thresh":
OLD: cat_thresh = [ <=CDP10, >CDP10&&<=CDP20, >CDP20&&<=CDP30, >CDP30&&<=CDP40, >CDP40&&<=CDP50, >CDP50&&<=CDP60, >CDP60&&<=CDP70, >CDP70&&<=CDP80, >CDP80&&<=CDP90, >CDP90 ];
NEW: obs_thresh = [ <=CDP10, >CDP10&&<=CDP20, >CDP20&&<=CDP30, >CDP30&&<=CDP40, >CDP40&&<=CDP50, >CDP50&&<=CDP60, >CDP60&&<=CDP70, >CDP70&&<=CDP80, >CDP80&&<=CDP90, >CDP90 ];

After making that change, Ensemble-Stat runs without error and produces output for each of the specified bins.

Here's the longer explanation as to why:
The "obs_thresh" option is a threshold which defines how to FILTER the pairs during the verification step. This option is parsed for each "obs.field" entry. Since you're setting defining it in "ens", and then copying "fcst = ens;" and "obs = fcst", it is getting set in the "obs" dictionary, and that's where Ensemble-Stat is parsing it.

This same logic can be done in Grid-Stat and Point-Stat, but in a much easier way... just define the number of climo bins you want to use in the config file:
climo_cdf = {
cdf_bins = 1;
center_bins = FALSE;
write_bins = TRUE;
}

Since this is exactly what you're trying to do in Ensemble-Stat, it seems like I should port that same logic from Point/Grid-Stat over to Ensemble-Stat.

@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing component: library code requestor: NOAA/EMC NOAA Environmental Modeling Center priority: blocker Blocker labels Dec 11, 2019
@JohnHalleyGotway JohnHalleyGotway added this to the MET 9.0 milestone Dec 11, 2019
@JohnHalleyGotway JohnHalleyGotway self-assigned this Dec 11, 2019
@JohnHalleyGotway
Copy link
Collaborator Author

Ensemble-Stat writes output lines types for ECNT, RHIST, PHIST, ORANK, SSVAR, and RELP.

I need to decide for which of these line types does it make sense to compute and report the mean across the bins.

I think it makes sense for ECNT.
I doesn't apply to the ORANK line type, and I can't do it for the RHIST line type since that contains integers. I also don't think that it makes sense for SSVAR.
I could technically do it for the PHIST and RELP line types, but I think the result would be pretty difficult to interpret.

@JohnHalleyGotway
Copy link
Collaborator Author

JohnHalleyGotway commented Dec 30, 2019

Completed development for this issue and committed to the feature branch. I tweaked an existing unit test to exercise this functionality. Merged to develop. The change will cause the NB to fail.

@JohnHalleyGotway
Copy link
Collaborator Author

Verified the NB diffs are as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: Library Code priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center type: enhancement Improve something that it is currently doing
Projects
None yet
Development

No branches or pull requests

1 participant