Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reader for Goes15 netcdf Eumetsat format #530

Merged
merged 30 commits into from
Dec 7, 2018

Conversation

TAlonglong
Copy link
Collaborator

@TAlonglong TAlonglong commented Nov 30, 2018

Adding in the nc_goes reader possibilities to read the GOES15 EUM format which has slightly different file names and geo location in a separate file equal for all data channels. Also the EUM format data is calibrated to Albedo for the VIS channel and to radiance for the IR channels

logger.debug('Calibration time: {}'.format(datetime.now() - tic))

# Mask space pixels
#data = data.where(self.meta['earth_mask'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E265 block comment should start with '# '

Returns:
Radiance [W m-2 um-1 sr-1]
"""
#rad = counts * slope + offset
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E265 block comment should start with '# '

logger.debug('Calibrating to reflectance')
# FIXME: quick hack commented out here as VIS is aleady calivrated to
# Albdo
#refl = 100 * k * radiance
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E265 block comment should start with '# '

# FIXME: quick hack commented out here as VIS is aleady calivrated to
# Albdo
#refl = 100 * k * radiance
#refl = k
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E265 block comment should start with '# '

except ValueError:
return r.text
for x in r.findall("./*"):
print x, x.tag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E999 SyntaxError: invalid syntax

res = res.where(res != res.attrs['_FillValue'])
res.attrs['_FillValue'] = np.nan


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

W293 blank line contains whitespace

res.attrs['_FillValue'] = np.nan


print "DATA:", self.nc[key.name]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E303 too many blank lines (2)

print "DATA:", self.nc[key.name]
print "END"

#print self.nc.attrs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E265 block comment should start with '# '

if 'missionName' in self.nc.attrs:
res.attrs.update({'platform_name': self.nc.attrs['missionName']})

print "res.shape: ",res.shape
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E231 missing whitespace after ','

@@ -196,9 +197,13 @@ def _channel_names(self, channels, cns, **kwargs):
return _image_description

def _add_sizes(self, datasets, first_dataset):
print "datasets.sizes: ", datasets.sizes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E999 SyntaxError: invalid syntax

@TAlonglong TAlonglong closed this Nov 30, 2018
@TAlonglong
Copy link
Collaborator Author

I have messed up several changes.

@TAlonglong TAlonglong reopened this Nov 30, 2018
# not needed this will be popped here.
if 'file_type' in info:
info.pop('file_type')

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

W293 blank line contains whitespace

@TAlonglong
Copy link
Collaborator Author

This need to be refactorized by me and @sfinkens before it can be approved.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.4%) to 74.198% when pulling a2538a0 on TAlonglong:goes15-nc-eum into 17b88e4 on pytroll:master.

@coveralls
Copy link

coveralls commented Nov 30, 2018

Coverage Status

Coverage decreased (-0.4%) to 74.198% when pulling a2538a0 on TAlonglong:goes15-nc-eum into 17b88e4 on pytroll:master.

@codecov
Copy link

codecov bot commented Nov 30, 2018

Codecov Report

Merging #530 into master will increase coverage by 0.13%.
The diff coverage is 89.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #530      +/-   ##
==========================================
+ Coverage   74.62%   74.75%   +0.13%     
==========================================
  Files         136      136              
  Lines       18342    18517     +175     
==========================================
+ Hits        13687    13842     +155     
- Misses       4655     4675      +20
Impacted Files Coverage Δ
satpy/tests/reader_tests/test_nc_goes.py 98.13% <100%> (+1.05%) ⬆️
satpy/tests/test_yaml_reader.py 98.77% <100%> (+0.12%) ⬆️
satpy/readers/yaml_reader.py 89.14% <100%> (+1.11%) ⬆️
satpy/readers/nc_goes.py 65.73% <75%> (-0.24%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed9bd67...4263fa6. Read the comment docs.

@@ -523,6 +558,17 @@ def __init__(self, filename, filename_info, filetype_info):
nlines=self.nlines,
ncols=self.ncols)
self._meta = None
self.geo_data = geo_data if geo_data is not None else self.nc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

W293 blank line contains whitespace

Trygve Aspenes and others added 5 commits December 4, 2018 13:31
# Instantiate reader using the mocked open_dataset() method
self.reader = GOESEUMNCFileHandler(filename='dummy', filename_info={},
filetype_info={}, geo_data=geo_data)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

W293 blank line contains whitespace

calibration=calib, channel=ch)
target_func.assert_called()

class GOESNCEUMFileHandlerReflectanceTest(unittest.TestCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E302 expected 2 blank lines, found 1

@TAlonglong
Copy link
Collaborator Author

I think this is ready for review @sfinkens . I don't understand why the tests dont pass.

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Pytroll Contribution Week at Eumetsat automation moved this from Needs review to Reviewer approved Dec 7, 2018
@mraspaud mraspaud changed the title Goes15 nc eum Add reader for Goes15 netcdf Eumetsat format Dec 7, 2018
@mraspaud mraspaud added enhancement code enhancements, features, improvements component:readers PCW Pytroll Contributors' Week labels Dec 7, 2018
@mraspaud mraspaud merged commit 8825f67 into pytroll:master Dec 7, 2018
Pytroll Contribution Week at Eumetsat automation moved this from Reviewer approved to Done Dec 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:readers enhancement code enhancements, features, improvements PCW Pytroll Contributors' Week
Development

Successfully merging this pull request may close these issues.

Support GOES-15 in netcdf format from Eumetcast (nc_goes reader)
5 participants