Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance METcalcpy to use the TOTAL_DIR column when aggregate statistics wind direction statistics in the VL1L2, VAL1L2, and VCNT columns #384

Open
7 of 22 tasks
JohnHalleyGotway opened this issue May 21, 2024 · 0 comments · Fixed by #389
Open
7 of 22 tasks

Enhance METcalcpy to use the TOTAL_DIR column when aggregate statistics wind direction statistics in the VL1L2, VAL1L2, and VCNT columns #384

JohnHalleyGotway opened this issue May 21, 2024 · 0 comments · Fixed by #389
Assignees
@bikegeek
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue METcalcpy: General Statistics priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Milestone
METcalcpy-3.0.0

Comments

@JohnHalleyGotway
Copy link
Contributor

JohnHalleyGotway commented May 21, 2024 •
edited
Loading

Describe the Enhancement

Issue dtcenter/METdataio#307 updates the schema to load the new TOTAL_DIR column from the VL1L2, VAL1L2, and VCNT line types added during the beta5 development cycle for MET version 12.0.0. This issue is to update METcalcpy to refine the aggregation logic for the VL1L2, VAL1L2, and VCNT line types. When aggregating statistics from those line types beginning with DIR_ME, DIR_MAE, or DIR_MSE use the counts in the TOTAL_DIR column to define the weights rather than those found in the TOTAL column, which is the existing logic.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center type: enhancement Improve something that it is currently doing alert: NEED ACCOUNT KEY Need to assign an account key to this issue required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone METcalcpy: General Statistics labels May 21, 2024
@JohnHalleyGotway JohnHalleyGotway added this to the METcalcpy-3.0.0 milestone May 21, 2024
@JohnHalleyGotway JohnHalleyGotway mentioned this issue May 21, 2024
Merged
17 tasks
bikegeek added a commit that referenced this issue Jun 22, 2024
@bikegeek
issue #384 tests for get_met_version code
ccf0686
@bikegeek bikegeek linked a pull request Jun 28, 2024 that will close this issue
Feature 384 agg stat total dir #389
Merged
15 tasks
hankenstein2 pushed a commit that referenced this issue Jun 28, 2024
@bikegeek
Feature 384 agg stat total dir (#389)
d7631db 
* issue #384 tests for get_met_version code

* Added code to calculate the total_dir and determining MET version

* Fixed import of utils module for get_met_version

* Update the vl1l2 data that was produced by the MET stat_analysis tool on vl1l2 data with the total_dir column

* updated config file for vl1l2 total_dir column addition

* modify test_vl1l2 for new column, TOTAL_DIR in linetype

* skip tests for val1l2 and vcnt until new data is being used

* Delete test/data/stat_analysis/point_stat_GRIB1_NAM_GDAS_120000L_20120409_120000V_vl1l2.txt

out of date now that TOTAL_DIR column has been added

* Delete test/vl1l2_agg_stat.yaml

out of date with addition of the TOTAL_DIR column

* Delete test/data/stat_analysis/met_vl1l2_agg.txt

out of date, no longer relevant

* Updated data with TOTAL_DIR column

* updated VL1L2 data with TOTAL_DIR column

* skip test_total_dir test, not yet finished

* Fix directory path

* Fixed more incorrect data paths

* skip the test_total_dir test, data needs to be updated and old files were removed, causing error messages about no files found

* Delete test/data/point_stat/met_v12/point_stat_GRIB2_NAM_NDAS_120000L_20120409_120000V_vcnt.txt

incorrect file

* updates for supporting the TOTAL_DIR column

* Modify path for input data

* udated VAL1L2 data aggregated via MET stat_analysis, contains the TOTAL_DIR column

* support for new TOTAL_DIR column in VAL1L2 linetype for calculating the associated ME, MAE, and MSE stats

* Updates to test for VAL1L2 linetype with new TOTAL_DIR column

* updated VAL1L2 data with TOTAL_DIR column

* Delete test/data/stat_analysis/point_stat_GRIB1_NAM_GDAS_120000L_20120409_120000V_val1l2.txt

out of date

* Delete test/data/stat_analysis/met_val1l2_stat_anal.txt

created with out of date MET data

* update calculation of the dir_xyz stats to use the TOTAL_DIR data

* updated data and tests for VCNT linetype with TOTAL_DIR column

* Updated VCNT data with new TOTAL_DIR column

* Delete Data/dummy_vl1l2.txt

not used

* Delete test/data/stat_analysis/point_stat_GRIB1_NAM_GDAS_120000L_20120409_120000V_vcnt.txt

out of date

* Delete test/data/stat_analysis/met_vcnt_from_vl1l2_aggstat.txt

out of date
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bikegeek bikegeek

Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue METcalcpy: General Statistics priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Projects
METplus-Analysis-6.0.0 Development
Status: ✅ Done
Milestone
METcalcpy-3.0.0
Development

Successfully merging a pull request may close this issue.

Feature 384 agg stat total dir dtcenter/METcalcpy
2 participants
@bikegeek @JohnHalleyGotway