Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using cfgrib to open Stage-IV rainfall data #27

Closed
edougherty32 opened this Issue Nov 6, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@edougherty32
Copy link

edougherty32 commented Nov 6, 2018

Hi,

I was wondering if it's possible to use cfgrib to open Stage-IV GRIB rainfall data (downloaded here). I attempted this in Jupyter notebook:

cfgrib.open_dataset('/glade/scratch/doughert/stage_iv/2008/ST4.2008031900.01h')

And received the following error:

EcCodesError: ('Key/value not found (-10).', -10)

During handling of the above exception, another exception occurred:
..
KeyError: 'latitudes

I am guessing this is due to the structure of the Stage-IV data, which is seen by opening the dataset via the usual xarray function:

xr.open_dataset('/glade/scratch/doughert/stage_iv/2008/ST4.2008031900.01h', engine='pynio')

<xarray.Dataset>
Dimensions:                 (g5_x_0: 881, g5_y_1: 1121)
Coordinates:
    g5_lat_0                (g5_x_0, g5_y_1) float32 ...
    g5_lon_1                (g5_x_0, g5_y_1) float32 ...
Dimensions without coordinates: g5_x_0, g5_y_1
Data variables:
    g5_rot_2                (g5_x_0, g5_y_1) float32 ...
    VAR_237_GDS5_SFC_acc1h  (g5_x_0, g5_y_1) float32 ...

Where the coordinates and variables aren't the usual "latitude" and "longitude" I've seen in the examples of cfgrib so far. So does this mean this package would be unusable for Stage IV data?

@rabernat

This comment has been minimized.

Copy link

rabernat commented Nov 7, 2018

Perhaps related to #17?

There are many different flavors of grib files out there in the wild. It would be good to be clear about which ones that cfgrib is committed to supporting.

@alexamici

This comment has been minimized.

Copy link
Collaborator

alexamici commented Nov 7, 2018

@edougherty32 looks like I assumed ecCodes could provide geographic coordinates for gridType==polar_stereographic but didn't test it. And obviously it doesn't.

In current master the "Precipitation NCEP/EMC 4KM Gridded Data (GRIB) Stage IV Data" GRIB files now don't crash cfgrib, but they are opened with no geographic support 😞. Please test them and report if it fixes your problem. I will probably add a different issue to track gridType support by ecCodes but that would be a wontfix for cfgrib as we intend to rely on the underlying library for these things.

@rabernat regarding coordinate systems support the commitment is the one in the README:

Limitations:

  • ...
  • relys on ecCodes for the gridType handling.

More in general we aim of cfgrib is quite simple: to support all GRIB files that

  • can be decoded by ecCodes (back-end)
  • can be represented in xarray (front-end)
@alexamici

This comment has been minimized.

Copy link
Collaborator

alexamici commented Nov 7, 2018

@edougherty32 as a quick update: I had a quick conversation with the ecCodes developers and looks like polar_stereographic is supported, just not by the interface I am using. We are assessing what is the best way to proceed.

@alexamici alexamici self-assigned this Nov 7, 2018

@alexamici alexamici added the bug label Nov 7, 2018

@edougherty32

This comment has been minimized.

Copy link
Author

edougherty32 commented Nov 7, 2018

@alexamici Thanks for the update. For now, I will stick with the

xarray.open_dataset()

function, setting the engine to 'pynio' to read in the Stage-IV rainfall.

However, I look forward to any developments on the use of cfgrib to work with Stage-IV files, as it might be more efficient than just using xarray.

@alexamici

This comment has been minimized.

Copy link
Collaborator

alexamici commented Nov 8, 2018

Next release of ecCodes will enable polar_stereographic for cfgrib. We still do not have a strategy on what to do on system with old ecCodes but we track progress in #28 .

Closing the issue.

@alexamici alexamici closed this Nov 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.