New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when loading a UK MetOffice seasonal forecast #12

Closed
matteodefelice opened this Issue Aug 10, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@matteodefelice
Copy link

matteodefelice commented Aug 10, 2018

I have downloaded a GRIB from the CDS with the following API request:

CDS API request
import cdsapi

c = cdsapi.Client()

r = c.retrieve(
    'seasonal-monthly-single-levels',
    {
        'originating_centre':'ukmo',
        'variable':'2m_temperature',
        'product_type':'monthly_mean',
        'year':[
            '2014','2015'
        ],
        'month':[
            '06','07'
        ],
        'leadtime_month':'1',
        'format':'grib'
    })

r.download('download.grib')
But when loading with `cfgrib` with the line:
my_data = xarray_store.open_dataset('/path/to/download.grib')

I get:

ValueError                                Traceback (most recent call last)
<ipython-input-63-523d0bd9e50f> in <module>()
----> 1 my_data = xarray_store.open_dataset('/path/to/download.grib')

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/xarray_store.py in open_dataset(path, flavour_name, filter_by_keys, **kwargs)
    160         if k.startswith('encode_'):
    161             overrides[k] = kwargs.pop(k)
--> 162     store = GribDataStore.frompath(path, **overrides)
    163     return _open_dataset(store, **kwargs)
    164 

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/xarray_store.py in frompath(cls, path, flavour_name, **kwargs)
    102         config = flavour.pop('dataset', {}).copy()
    103         config.update(kwargs)
--> 104         return cls(ds=cfgrib.Dataset.frompath(path, **config), **flavour)
    105 
    106     def __attrs_post_init__(self):

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/dataset.py in frompath(cls, path, mode, **kwargs)
    393     @classmethod
    394     def frompath(cls, path, mode='r', **kwargs):
--> 395         return cls(stream=messages.Stream(path, mode, message_class=cfmessage.CfMessage), **kwargs)
    396 
    397     def __attrs_post_init__(self):

<attrs generated init c839a147c90eb3321ada82313ce86e3ade1b1758> in __init__(self, stream, encode_parameter, encode_time, encode_vertical, encode_geography, filter_by_keys)
      6     self.encode_geography = encode_geography
      7     self.filter_by_keys = filter_by_keys
----> 8     self.__attrs_post_init__()

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/dataset.py in __attrs_post_init__(self)
    396 
    397     def __attrs_post_init__(self):
--> 398         dims, vars, attrs = build_dataset_components(**self.__dict__)
    399         self.dimensions = dims  # type: T.Dict[str, T.Optional[int]]
    400         self.variables = vars  # type: T.Dict[str, Variable]

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/dataset.py in build_dataset_components(stream, encode_parameter, encode_time, encode_vertical, encode_geography, filter_by_keys)
    367         var_index = index.subindex(paramId=param_id)
    368         dims, data_var, coord_vars = build_data_var_components(
--> 369             var_index, encode_parameter, encode_time, encode_geography, encode_vertical,
    370         )
    371         if encode_parameter and var_name != 'undef':

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/dataset.py in build_data_var_components(index, encode_parameter, encode_time, encode_geography, encode_vertical, log)
    281     data_var_attrs_keys = DATA_ATTRIBUTES_KEYS[:]
    282     data_var_attrs_keys.extend(GRID_TYPE_MAP.get(index.getone('gridType'), []))
--> 283     data_var_attrs = enforce_unique_attributes(index, data_var_attrs_keys)
    284     if encode_parameter:
    285         data_var_attrs['standard_name'] = data_var_attrs.get('GRIB_cfName', 'undef')

~/miniconda2/envs/cds/lib/python3.6/site-packages/cfgrib/dataset.py in enforce_unique_attributes(index, attributes_keys)
    113         values = index[key]
    114         if len(values) > 1:
--> 115             raise ValueError("multiple values for unique attribute %r: %r" % (key, values))
    116         if values:
    117             attributes['GRIB_' + key] = values[0]

ValueError: multiple values for unique attribute 'longitudeOfFirstGridPointInDegrees': [0.0, 0.5]
@alexamici

This comment has been minimized.

Copy link
Collaborator

alexamici commented Aug 13, 2018

@matteodefelice that's interesting, it looks like the GRIB file contains two fields with different regular_ll grids for every date / leadtime / perturbationNumber combination, one centred in [0, 0] and the other in [0.5, 0.5], I have no idea if this is the intended behaviour.

You can use the filter_by_key work-around as documented in the Advanced Usage section of the README.

In this case:

>>> open_dataset('download.grib', filter_by_keys={'longitudeOfFirstGridPointInDegrees': 0})
<xarray.Dataset>
Dimensions:     (latitude: 180, longitude: 360, number: 28, step: 2, time: 4)
Coordinates:
  * number      (number) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ...
  * time        (time) datetime64[ns] 2014-06-01 2014-07-01 2015-06-01 ...
  * step        (step) timedelta64[ns] 30 days 31 days
    surface     int64 ...
  * latitude    (latitude) float64 89.0 88.0 87.0 86.0 85.0 84.0 83.0 82.0 ...
  * longitude   (longitude) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 ...
    valid_time  (time, step) datetime64[ns] ...
Data variables:
    t2m         (number, time, step, latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            1
    GRIB_centre:             egrr
    GRIB_centreDescription:  U.K. Met Office - Exeter
    GRIB_subCentre:          98
    history:                 GRIB to CDM+CF via cfgrib-0.8.4.3/ecCodes-2.7.0

and:

>>> open_dataset('download.grib', filter_by_keys={'longitudeOfFirstGridPointInDegrees': 0.5})
<xarray.Dataset>
Dimensions:     (latitude: 180, longitude: 360, number: 28, step: 2, time: 4)
Coordinates:
  * number      (number) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ...
  * time        (time) datetime64[ns] 2014-06-01 2014-07-01 2015-06-01 ...
  * step        (step) timedelta64[ns] 30 days 31 days
    surface     int64 ...
  * latitude    (latitude) float64 89.5 88.5 87.5 86.5 85.5 84.5 83.5 82.5 ...
  * longitude   (longitude) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 ...
    valid_time  (time, step) datetime64[ns] ...
Data variables:
    t2m         (number, time, step, latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            1
    GRIB_centre:             egrr
    GRIB_centreDescription:  U.K. Met Office - Exeter
    GRIB_subCentre:          98
    history:                 GRIB to CDM+CF via cfgrib-0.8.4.3/ecCodes-2.7.0

I consider this issue to be a duplicate, although a quite surprising one, of #2 , I'd close it and we can discuss on how to improve the current cfgrib behaviour there.

@alexamici alexamici closed this Aug 13, 2018

@alexamici alexamici added the duplicate label Aug 13, 2018

@alexamici alexamici self-assigned this Aug 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment