Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance MET NetCDF library code to handle time variables of type byte. #1493

Closed
8 of 20 tasks
JohnHalleyGotway opened this issue Sep 15, 2020 · 2 comments
Closed
8 of 20 tasks
Assignees
Labels
MET: Library Code priority: medium Medium Priority reporting: DTC NOAA R2O NOAA Research to Operations DTC Project type: enhancement Improve something that it is currently doing
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

Describe the Enhancement

This issue arose when attempting to read a new dataset into MET in preparation for physics testing in HAFS. Kathryn Newman is the contact person for this work.

CYGNSS data (level 3 surface wind data) in NetCDF format can be found here:
https://podaac.jpl.nasa.gov/dataset/CYGNSS_L3_V2.1

The NetCDF level 3 data is gridded and is on a lat/lon grid... so no projection details with which to contend. I grabbed a sample data file and ran:

plot_data_plane cyg.ddmi.s20200803-003000-e20200803-233000.l3.grid-wind.a10.d21.nc wind_speed.ps 'name="wind_speed"; level="(0,*,*)";'
ERROR  : 
ERROR  : get_double_var() -> data type mismatch (double vs. "byte").
ERROR  : It won't be converted because of dimension issue.
ERROR  : Please correct the data type to double for variable "time".
ERROR  :

After running ncdump/ncgen to manually change the time variable from type byte to double, plot_data_plane reads this data just fine and produces this image:
wind_speed

This task is to enhance the NetCDF library support in MET to handle the time variable being defined as type byte directly. A sample input NetCDF data file is attached.
cyg.ddmi.s20200803-003000-e20200803-233000.l3.grid-wind.a10.d21.nc.txt

Time Estimate

1 day.

Sub-Issues

Consider breaking the enhancement down into sub-issues.
No sub-issues required.

Relevant Deadlines

NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required: Howard
  • Select scientist(s) or no scientist required: Kathryn

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
  • Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project(s), Milestone, and Linked issues
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing component: library code priority: medium Medium Priority alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Sep 15, 2020
@JohnHalleyGotway JohnHalleyGotway added this to the MET 10.0 milestone Sep 15, 2020
@hsoh-u hsoh-u removed the alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle label Oct 7, 2020
@hsoh-u
Copy link
Collaborator

hsoh-u commented Oct 7, 2020

It's done for attribute (get_att_unixtime at nc_var_info.cc), but not for the data variable.

@hsoh-u
Copy link
Collaborator

hsoh-u commented Oct 13, 2020

It's different from handling of an time attribute which contains the string of "yyyymmdd" form. The time variable at the sample data is 1 byte integer (not character).

hsoh-u added a commit that referenced this issue Oct 20, 2020
hsoh-u added a commit that referenced this issue Oct 21, 2020
#1493 get_nc_att(NcFile) Do not display the error message if exit_on_error is not enabled
jprestop pushed a commit that referenced this issue Oct 21, 2020
* Check if the input file exists to avoid seg fault

* #1493 Renamed get_double_var to get_nc_time and supports 1 byte integer

* #1493 Added netcdf_1byte_time

* Remove commented-out development code.

* Removing commented-out development code.

* Update nc_utils.cc

#1493 get_nc_att(NcFile) Do not display the error message if exit_on_error is not enabled

Co-authored-by: johnhg <johnhg@ucar.edu>
@JohnHalleyGotway JohnHalleyGotway removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Oct 22, 2020
@TaraJensen TaraJensen added the reporting: DTC NOAA R2O NOAA Research to Operations DTC Project label Dec 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: Library Code priority: medium Medium Priority reporting: DTC NOAA R2O NOAA Research to Operations DTC Project type: enhancement Improve something that it is currently doing
Projects
None yet
Development

No branches or pull requests

4 participants