Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Use Case: Feature Relative using MTD output for feature centroid lat/lon #641

Closed
18 of 21 tasks
TaraJensen opened this issue Oct 1, 2020 · 4 comments · Fixed by #968 or #1153
Closed
18 of 21 tasks

New Use Case: Feature Relative using MTD output for feature centroid lat/lon #641

TaraJensen opened this issue Oct 1, 2020 · 4 comments · Fixed by #968 or #1153
Assignees
Labels
alert: NEED MORE DEFINITION Not yet actionable, additional definition required priority: high High Priority requestor: University/SBU Stony Brook University required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project type: enhancement Improve something that it is currently doing
Milestone

Comments

@TaraJensen
Copy link
Contributor

TaraJensen commented Oct 1, 2020

Describe the Enhancement

Currently the Feature Relative use-cases identify the location of the feature from output from the GFDL tracker. The SBU and UW-CIMSS projects would also like to have support for MTD centroid lat/lon. I think we need to discuss if this can be handled in the python script that provides lat/lons to the script that cuts out the tile prior to passing into Series-Analysis, or if it requires modifications to TC-pairs/TC-stat to allow for further refinement.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

December 2020

Funding Source

7791041

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
  • Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project(s), Milestone, and Linked issues
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@TaraJensen TaraJensen added type: enhancement Improve something that it is currently doing component: use case configuration priority: high High Priority alert: NEED MORE DEFINITION Not yet actionable, additional definition required labels Oct 1, 2020
@TaraJensen TaraJensen added this to the METplus-4.0 milestone Oct 1, 2020
@TaraJensen TaraJensen added this to To do in METplus-4.0.0-beta2 (12/8/20) via automation Oct 1, 2020
@georgemccabe georgemccabe added this to To do in METplus-4.0.0-beta3 (1/27/21) via automation Nov 19, 2020
@JohnHalleyGotway JohnHalleyGotway added requestor: University/SBU Stony Brook University and removed requestor:SBU labels Dec 11, 2020
@georgemccabe georgemccabe added this to To do in METplus-4.0.0-beta4 (3/2/21) via automation Jan 21, 2021
@georgemccabe
Copy link
Collaborator

The ExtractTiles wrapper (and optionally the SeriesAnalysis wrapper) reads a TCStat output file to get a list of storms and the lat/lon points along the track. The wrappers could be modified to read a MTD output file to parse out the lat/lon points from the appropriate columns. However, we will need to determine how to group these points.

@georgemccabe georgemccabe added this to To do in METplus-4.1.0-beta1 (7/22/21) via automation Feb 25, 2021
@TaraJensen TaraJensen added required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project alert: NEED ACCOUNT KEY Need to assign an account key to this issue labels May 17, 2021
@JohnHalleyGotway
Copy link
Collaborator

Here's some recommended logic for implementing this functionality in METplus. Please see this file to illustrate:
mtd_BASIC_20100517_010000V_2d.txt

The task here is to run Series-Analysis once for each pair of fcst/obs MTD space-time objects. So we need to construct a list of times along with fcst_lat, fcst_lon and obs_lat, obs_lon locations. This can be done by processing MTD files ending with _2d.txt. The rest of the MTD output files can be ignored.

  1. First, figure out how many fcst/obs MTD object pairs there are to process by processing the "OBJECT_CAT" column. They contain values of "CF###" or "CO###". The maximum value of those ### indicates the number of series there are to process. If they are all 000, then there is no work to do.
  2. Loop through the series of object numbers (001, 002, and so on), and for each...
  3. Store the forecast locations by retrieving the the "CENTROID_LAT" and "CENTROID_LON" columns from the lines where "OBJECT_ID = CF###" (where ### is the current number).
  4. Store the observations locations by retrieving the the "CENTROID_LAT" and "CENTROID_LON" columns from the lines where "OBJECT_ID = CO###" (where ### is the current number).

So for each timestep, we're storing the location of the centroid for the 2D cluster object of the forecast and the observation. My only question is whether or not the number of these time slices can differ. In the attached example, they correspond 1-to-1, but it seems conceivable to me that one could begin/end earlier/later than the other, making it so that the timesteps don't exactly correspond.

@TaraJensen TaraJensen moved this from To do to In progress in METplus-4.1.0-beta1 (7/22/21) Jun 7, 2021
@TaraJensen TaraJensen removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Jun 7, 2021
@georgemccabe georgemccabe linked a pull request Jun 28, 2021 that will close this issue
12 tasks
@DanielAdriaansen
Copy link
Contributor

We will need to utilize the work in #968 to enhance existing feature relative use cases, but I'm not sure which ones those are. @TaraJensen may be able to help. We can either leave this issue open, or close it and open new issues for enhancing the use cases.

@georgemccabe georgemccabe removed this from In progress in METplus-4.1.0-beta1 (7/22/21) Jul 1, 2021
@georgemccabe georgemccabe added this to To do in METplus-4.1.0-beta2 (8/31/21) via automation Jul 1, 2021
@TaraJensen TaraJensen changed the title Enhance Feature Relative Use-Case to work with MTD output for feature centroid lat/lon Develop Feature Relative Use-Case to work with MTD output for feature centroid lat/lon Jul 26, 2021
@JohnHalleyGotway JohnHalleyGotway moved this from To do to In progress in METplus-4.1.0-beta2 (8/31/21) Aug 17, 2021
@georgemccabe georgemccabe removed this from In progress in METplus-4.1.0-beta2 (8/31/21) Aug 31, 2021
@georgemccabe georgemccabe added this to To do in METplus-4.1.0-beta3 (10/06/21) via automation Aug 31, 2021
@DanielAdriaansen DanielAdriaansen linked a pull request Sep 13, 2021 that will close this issue
12 tasks
@georgemccabe georgemccabe moved this from To do to Pull Request Review in METplus-4.1.0-beta3 (10/06/21) Sep 28, 2021
@georgemccabe
Copy link
Collaborator

Closed with #1153

METplus-4.1.0-beta3 (10/06/21) automation moved this from Pull Request Review to Done Sep 30, 2021
@georgemccabe georgemccabe changed the title Develop Feature Relative Use-Case to work with MTD output for feature centroid lat/lon New Use Case: Feature Relative using MTD output for feature centroid lat/lon Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED MORE DEFINITION Not yet actionable, additional definition required priority: high High Priority requestor: University/SBU Stony Brook University required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project type: enhancement Improve something that it is currently doing
Projects
No open projects
6 participants