Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for inconsistent setting of flags (-D__64BIT__) in MET and the GRIB2C library #1416

Closed
18 tasks done
jprestop opened this issue Jul 16, 2020 · 2 comments
Closed
18 tasks done
Assignees
Labels
MET: Library Code priority: medium Medium Priority requestor: NCAR National Center for Atmospheric Research type: enhancement Improve something that it is currently doing
Milestone

Comments

@jprestop
Copy link
Collaborator

jprestop commented Jul 16, 2020

Describe the Enhancement

When the GRIB2 flags are set inconsistently, MET’s GRIB2 library code thinks it needs to read a ridiculously large amount of memory. We want to find that spot, add a sanity check, and print a warning message message if requesting an unreasonable amount of memory. Don't make it an error message on the off chance that whatever size threshold we choose is too small for some user. This issue comes up often enough in met_help issues where it seems worthwhile to add a check in the code.

Can test using the data in /d3/personal/jpresto/met_help/shilpi_kalra_20200712.
Run: pcp_combine -subtract WRFPRS_d01.48 48 WRFPRS_d01.24 24 APCP.nc
Compile a version of MET where there is inconsistent setting of the -D__64BIT__ flag between the GRIB2C library and MET.

Time Estimate

4 hours.

Sub-Issues

No sub-issues needed.

Relevant Deadlines

NONE

Funding Source

NONE

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required: John Halley Gotway
  • Select scientist(s) or no scientist required: No scientist needed

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones
  • Select milestone

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@jprestop jprestop added type: enhancement Improve something that it is currently doing component: library code priority: medium Medium Priority requestor: NCAR National Center for Atmospheric Research alert: NEED ACCOUNT KEY Need to assign an account key to this issue labels Jul 16, 2020
@jprestop jprestop added this to To do in MET-9.1-beta3 (7/29/20) via automation Jul 17, 2020
@jprestop jprestop added this to the MET 9.1 milestone Jul 17, 2020
@JohnHalleyGotway JohnHalleyGotway moved this from To do to In progress in MET-9.1-beta3 (7/29/20) Jul 20, 2020
@JohnHalleyGotway JohnHalleyGotway self-assigned this Jul 20, 2020
@JohnHalleyGotway JohnHalleyGotway changed the title Check for inconsistent setting of flags ( -D__64BIT__ ) in MET and the GRIB2C library Check for inconsistent setting of flags (-D__64BIT__) in MET and the GRIB2C library Jul 21, 2020
@JohnHalleyGotway JohnHalleyGotway removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Jul 21, 2020
@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Jul 21, 2020

Testing on dakota in /d3/personal/johnhg/MET/MET_development/MET-feature_1416_64bit.
I had been compiling on dakota using:

CFLAGS=-g -DUNDERSCORE -fPIC -D__64BIT__
CXXFLAGS=-g -DUNDERSCORE -fPIC -D__64BIT__

For this work, I removed the -D__64BIT__ flag from these settings and recompiled and tested by running:

/d3/personal/johnhg/MET/MET_development/MET-feature_1416_64bit/met/bin/pcp_combine -subtract \
/d3/personal/jpresto/met_help/shilpi_kalra_20200712/WRFPRS_d01.48 48 \
/d3/personal/jpresto/met_help/shilpi_kalra_20200712/WRFPRS_d01.24 24 \
APCP.nc

ERROR  : 
ERROR  : out of memory!  Exiting!
ERROR  :

@JohnHalleyGotway
Copy link
Collaborator

Added a new oom_grib2() function and call set_new_handler() to point to it inside the GRIB2 library code. Retest to produce this new out of memory error message:

ERROR  : 
ERROR  : Out of memory reading GRIB2 data!  Exiting!
ERROR  : Check that MET and the GRIB2C library were compiled consistently, either with or without the -D__64BIT__ flag.
ERROR  : 

JohnHalleyGotway added a commit that referenced this issue Jul 21, 2020
…sage. Update the grid_grib2_record() function to call set_new_handler() to point to oom_grib2()... and be sure to switch back to the default oom function before exiting.
@JohnHalleyGotway JohnHalleyGotway moved this from In progress to Pull request review in MET-9.1-beta3 (7/29/20) Jul 21, 2020
@JohnHalleyGotway JohnHalleyGotway moved this from Pull request review to Done in MET-9.1-beta3 (7/29/20) Jul 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: Library Code priority: medium Medium Priority requestor: NCAR National Center for Atmospheric Research type: enhancement Improve something that it is currently doing
Projects
No open projects
Development

No branches or pull requests

2 participants