Genesis event definition criteria is not being applied to BEST and operational tracks.

[JohnHalleyGotway]

Describe the Problem

This is a bug that Dan Adriaansen found in his testing of tc_gen. The TC-Gen configuration file includes genesis event definition criteria for forecast tracks, BEST tracks, and operational tracks. However, only the forecast event definition criteria is being applied. When adding more stringent criteria for BEST and operational tracks, the number of identified genesis events remain the same.

Expected Behavior

The BEST and operational genesis event criteria should be applied as specified in the tc_gen configuration file.

Environment

Describe your runtime environment: Should be reproducible across all platforms.
1. Hardware: Mac Laptop
2. OS: MacOS
3. tc_gen in MET version 9.0

To Reproduce

Describe the steps to reproduce the behavior:
1. Download the attached sample ATCF file:
bsh922016.dat.txt

2. Download the attached TCGenConfig file:
TCGenConfig.txt

3. Run the following command using met-9.0 and note the log message:

tc_gen -genesis bsh922016.dat.txt -track bsh922016.dat.txt -config TCGenConfig.txt
DEBUG 2: Identified 1 analysis genesis events.

4. No genesis should be identified for 3 reasons:

best_genesis = {
   technique   = "BEST";
   category    = [ "HU" ];   # Input is "TD"
   vmax_thresh = >=31;    # Input has vmax = 30
   mslp_thresh = <=1000; # Input has mslp = 1009
}

Relevant Deadlines

None.

Funding Source

None.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: John Halley Gotway
• Select scientist(s) or no scientist required: Dan Adriaansen (will review the PR)

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Review projects and select relevant Repository and Organization ones
• Select milestone

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdb, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above.
• Fork this repository or create a branch of master_<Version>.
  Branch name: bugfix_<Issue Number>_master_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into master_<Version>.
  Pull request: bugfix <Issue Number> master_<Version> <Description>
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
• Close this issue.

[JohnHalleyGotway]

Merged changes into the develop branch. Will not merge changes into the master_v9.0 branch.

[JohnHalleyGotway]

After merging PR #1428 for this bugfix into the develop branch the nightly build failed on dakota. The tc_gen counts changed, moving some hits over to the false alarm column. I investigated the differences and proved to myself that they are correct. Below is an excerpt of an email describing the comparison:

My explanation about the BEST track data was wrong. All the BEST tracks in dakota:/d3/projects/MET/MET_test_data/unit_test/tc_data/genesis/atcf/2016/b* reached the TD or TS level, as you'd expect. The difference is in the CARQ events, as seen in this tc_gen log message:

OLD...

DEBUG 2: [Filter 1 (AL_BASIN) : Model 1] For SUI1 model, comparing 503 genesis forecasts to 17 BEST and 618 CARQ genesis events.

NEW...

DEBUG 2: [Filter 1 (AL_BASIN) : Model 1] For SUI1 model, comparing 503 genesis forecasts to 17 BEST and 227 CARQ genesis events.

Here's the operational track filtering criteria that also was not being applied:

oper_genesis = {
   technique   = "CARQ";
   category    = [ "DB", "LO", "WV" ];
   vmax_thresh = NA;
   mslp_thresh = NA;
}

Looking at the input ATCF track data, there are 1372 CARQ and 39 BEST tracks = 1411

cat /d3/projects/MET/MET_test_data/unit_test/tc_data/genesis/atcf/2016/a*.dat | grep CARQ | awk '{print $1, $2, $3, $4, $5}' | sort -u | wc -l
1372
cat /d3/projects/MET/MET_test_data/unit_test/tc_data/genesis/atcf/2016/b*.dat | grep BEST | awk '{print $1, $2}' | sort -u | wc -l
39

The OLD version has a log message listing that exact unfiltered count:

DEBUG 2: Identified 1411 analysis genesis events.

The NEW version log message lists this count:

DEBUG 2: Identified 504 analysis genesis events.

The 504 would be 39 BEST tracks plus 465 CARQ tracks. And that's exactly what I get I run this command below:

cat /d3/projects/MET/MET_test_data/unit_test/tc_data/genesis/atcf/2016/a*.dat | grep CARQ | awk '{print $1, $2, $3, $4, $5, $6, $11}' | grep ', 0,' | sort -u  | egrep "DB,|LO,|WV," | wc -l
465

So I'm confident now that the filters are being applied the way they're specified in the configuration file. And we have some hits from the old version that move to false alarms in the new version because the previously matching CARQ event is now filtered out.

But Kathryn and Dan H, can you please confirm that the settings are correct? When matching against CARQ genesis events is setting category = [ "DB", "LO", "WV" ]; the appropriate default setting? Below, I've broken the 1372 CARQ tracks down by intensity level:

DB = 180
EX = 13
HU = 261
LO = 280
SS = 2
TD = 160
TS = 471
WV = 5

Does is make sense scientifically to only include the DB, LO, and WV ones?