Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix 3a3b transition in the aapp l1b reader #805

Merged
merged 5 commits into from
Jun 10, 2019
Merged

Fix 3a3b transition in the aapp l1b reader #805

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
084acf4
Merge pull request #10 from pytroll/master
adybbroe May 3, 2019
95c2864
Merge branch 'master' of https://github.com/pytroll/satpy
adybbroe May 28, 2019
330f13a
Merge branch 'master' of https://github.com/pytroll/satpy
adybbroe May 29, 2019
01bf277
Merge branch 'master' of https://github.com/pytroll/satpy
adybbroe Jun 5, 2019
7445f0e
Masking for both 3a and 3b, avoiding data in the transition all together
adybbroe Jun 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions satpy/readers/aapp_l1b.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Copyright (c) 2012, 2013, 2014, 2015, 2016, 2017
# Copyright (c) 2012 - 2019, Pytroll

# Author(s):

Expand Down Expand Up @@ -81,6 +81,7 @@ def __init__(self, filename, filename_info, filetype_info):
self._data = None
self._header = None
self._is3b = None
self._is3a = None
self._shape = None
self.lons = None
self.lats = None
Expand Down Expand Up @@ -237,11 +238,12 @@ def calibrate(self,

if dataset_id.name in ("3a", "3b") and self._is3b is None:
# Is it 3a or 3b:
is3b = np.expand_dims(
np.bitwise_and(
np.right_shift(self._data['scnlinbit'], 0), 1) == 1, 1)
is3b = np.expand_dims(np.bitwise_and(self._data['scnlinbit'], 3) == 1, 1)
self._is3b = np.repeat(is3b,
self._data['hrpt'][0].shape[0], axis=1)
is3a = np.expand_dims(np.bitwise_and(self._data['scnlinbit'], 3) == 0, 1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice that these operations don't seem to be dask at all. As such, could we only do this bitwise_and once? Right now, it is used for both 3b and 3a checks. Or, even better, convert this reader to use dask better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we discussed that here. @mraspaud indicated that this could need a daskification overhaul. I just wanted to make sure we get this improvement in now. The Dask could come in the next step. Also, there is no test for this reader it seems, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. It seems there are no tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will see what I might have/take time to fix. Both concerning Dask and tests. But, feel under pressure fixing our transition to our Satpy based production, and this is a minor thing in this regard...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so I will merge this now, given the time constraints we have on this being fixed and the time it would take to overhaul this (tests and dask-friendliness) and we open an issue to remind us on what needs to be done.

self._is3a = np.repeat(is3a,
self._data['hrpt'][0].shape[0], axis=1)

try:
vis_idx = ['1', '2', '3a'].index(dataset_id.name)
Expand All @@ -258,7 +260,7 @@ def calibrate(self,
dataset_id.calibration,
pre_launch_coeffs,
coeffs,
mask=(dataset_id.name == '3a' and self._is3b)))
mask=(dataset_id.name == '3a' and np.logical_not(self._is3a))))
else:
ds = create_xarray(
_ir_calibrate(self._header,
Expand Down