Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GRIB reader for greater flexibility. #1586

Merged
merged 3 commits into from Mar 11, 2021

Conversation

simonrp84
Copy link
Member

Currently, the grib reader expects certain keys to be present in the grib file, for example:

ds_info.update({
            'filename': self.filename,
            'shortName': msg['shortName'],
            'long_name': msg['name'],
            'pressureUnits': msg['pressureUnits'],
            'typeOfLevel': msg['typeOfLevel'],
            'standard_name': msg['cfName'],
            'units': msg['units'],
            'modelName': msg['modelName'],
            'model_time': model_time,
            'centreDescription': center_description,
            'valid_min': msg['minimum'],
            'valid_max': msg['maximum'],
            'start_time': start_time,
            'end_time': end_time,
            'sensor': msg['modelName'],
            # National Weather Prediction
            'platform_name': 'unknown',
        })

For some grib files, such as those from the ERA5 and CAMS datasets, not all keys are present. modelName is not present in ERA5 single level datasets.

This PR adds some flexibility to the grib reader and will now set missing keys to unknown in ds_info rather than raising an error.
I have only tested this with ECMWF/ERA/GFS data but it seems to work well with those.

  • Tests passed
  • Passes flake8 satpy

…l attributes. Also update tests to reflect this and add some additional tests.
@codecov
Copy link

codecov bot commented Mar 5, 2021

Codecov Report

Merging #1586 (f071cb9) into master (f248972) will decrease coverage by 0.18%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1586      +/-   ##
==========================================
- Coverage   92.84%   92.66%   -0.19%     
==========================================
  Files         251      253       +2     
  Lines       37193    37199       +6     
==========================================
- Hits        34532    34470      -62     
- Misses       2661     2729      +68     
Flag Coverage Δ
behaviourtests 4.78% <0.00%> (+0.30%) ⬆️
unittests 92.80% <100.00%> (-0.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
satpy/readers/grib.py 84.61% <100.00%> (+2.12%) ⬆️
satpy/tests/reader_tests/test_grib.py 97.79% <100.00%> (+0.38%) ⬆️
satpy/tests/utils.py 92.61% <0.00%> (-5.09%) ⬇️
satpy/readers/__init__.py 96.99% <0.00%> (-0.43%) ⬇️
satpy/tests/test_scene.py 99.43% <0.00%> (-0.30%) ⬇️
satpy/tests/reader_tests/test_mirs.py 100.00% <0.00%> (ø)
satpy/readers/mirs.py 71.93% <0.00%> (ø)
satpy/tests/writer_tests/test_cf.py 99.69% <0.00%> (+<0.01%) ⬆️
satpy/tests/test_dependency_tree.py 98.24% <0.00%> (+0.20%) ⬆️
satpy/dataset/dataid.py 94.27% <0.00%> (+0.22%) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f248972...f071cb9. Read the comment docs.

satpy/readers/grib.py Outdated Show resolved Hide resolved
Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mraspaud mraspaud added component:readers enhancement code enhancements, features, improvements labels Mar 11, 2021
@mraspaud mraspaud added this to the v0.26.0 milestone Mar 11, 2021
@ghost
Copy link

ghost commented Mar 11, 2021

Congratulations 🎉. DeepCode analyzed your code in 588.8 seconds and we found no issues. Enjoy a moment of no bugs ☀️.

👉 View analysis in DeepCode’s Dashboard | Configure the bot

Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes. Will merge when tests pass. We should make sure all the other PRs are closed when it is merged.

@djhoese djhoese merged commit c74c396 into pytroll:master Mar 11, 2021
@simonrp84 simonrp84 deleted the grib-update branch March 11, 2021 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:readers enhancement code enhancements, features, improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants