Fix hrit-goes reader

[mraspaud]

The hrit goes reader is now calibrating and masking data correctly

• Closes #xxxx
• Tests added
• Tests passed
• Passes git diff origin/develop **/*py | flake8 --diff
• Fully documented

[djhoese]

Nice, this should increase coverage ;)

[coveralls]

Coverage increased (+0.4%) to 65.875% when pulling 8e27d3b on fix-hrit-geos-reader into 8a4d74f on develop.

[codecov]

Codecov Report

Merging #288 into develop will decrease coverage by 0.03%.
The diff coverage is 59.43%.

@@             Coverage Diff             @@
##           develop     #288      +/-   ##
===========================================
- Coverage    65.48%   65.44%   -0.04%     
===========================================
  Files          117      118       +1     
  Lines        14273    14772     +499     
===========================================
+ Hits          9347     9668     +321     
- Misses        4926     5104     +178

Impacted Files	Coverage Δ
satpy/readers/hrit_msg.py	25.23% <ø> (+0.34%)	⬆️
satpy/readers/hrit_base.py	76% <10%> (-5.74%)	⬇️
satpy/resample.py	50.15% <100%> (ø)	⬆️
satpy/tests/reader_tests/__init__.py	96.42% <100%> (+0.13%)	⬆️
satpy/readers/hrit_goes.py	50.68% <11.76%> (+23.35%)	⬆️
satpy/tests/reader_tests/test_hrit_goes.py	75.32% <75.32%> (ø)
satpy/readers/utils.py	63.63% <0%> (-2.6%)	⬇️
satpy/readers/nc_nwcsaf.py	16.17% <0%> (-0.85%)	⬇️
satpy/composites/__init__.py	32.25% <0%> (-0.68%)	⬇️
... and 5 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a4d74f...8e27d3b. Read the comment docs.

[AranJM]

Does not seem to work for me.
Tried the following:

from glob import glob
from satpy.scene import Scene

files = glob('/data1/storm/geos/tmp/*GOES15*10_7*140600*')
scn = Scene(filenames=files, reader='hrit_goes')
scn.load(['10_7'])

scn_NA = scn.resample('northamerica')

Result:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-ec6db78ae980> in <module>()
----> 1 scn_NA = scn.resample('northamerica')

/usr/local/lib/python2.7/dist-packages/satpy/scene.pyc in resample(self, destination, datasets, generate, unload, **resample_kwargs)
    741             destination = self.max_area(to_resample)
    742         new_scn, destination_area = self._resampled_scene(to_resample, destination,
--> 743                                                           **resample_kwargs)
    744
    745         new_scn.attrs = self.attrs.copy()

/usr/local/lib/python2.7/dist-packages/satpy/scene.pyc in _resampled_scene(self, datasets, destination, **resample_kwargs)
    706                 slice_x, slice_y = source_area.get_area_slices(destination_area)
    707                 source_area = source_area[slice_y, slice_x]
--> 708                 dataset.data = dataset.data.rechunk(1024)
    709                 dataset = dataset.isel(x=slice_x, y=slice_y)
    710                 dataset.attrs['area'] = source_area

AttributeError: 'numpy.ndarray' object has no attribute 'rechunk'

Tried with a different destination, which gave the same result.

Edit: scene has not been changed though. Could it be environment issue? Numpy version is 1.14.3

[mraspaud]

@AranJM I can't reproduce your error, are you sure you are on the right branch ?

[AranJM]

I do:
sudo pip install git+https://github.com/pytroll/satpy.git@fix-hrit-geos-reader
and among the output I get:
Successfully installed satpy-0.9.0a2.dev0

[djhoese]

@AranJM Try doing sudo pip install -U ... to update the package and/or show us the entire pip install output. Additionally, you should really use virtual environments or conda to avoid installing things to your system python as root/sudo.

[AranJM]

Same result when using -U flag.

Complete output from install:

The directory '/home/min_adm/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/min_adm/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting git+https://github.com/pytroll/satpy.git@fix-hrit-geos-reader
  Cloning https://github.com/pytroll/satpy.git (to revision fix-hrit-geos-reader) to /tmp/pip-req-build-OC4uma
Requirement not upgraded as not directly required: numpy>=1.4.1 in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (1.14.3)
Requirement not upgraded as not directly required: pillow in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (5.1.0)
Requirement not upgraded as not directly required: pyresample>=1.9.1 in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (1.9.1)
Requirement not upgraded as not directly required: trollsift in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (0.2.0)
Requirement not upgraded as not directly required: trollimage>=1.5.1 in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (1.5.1)
Requirement not upgraded as not directly required: pykdtree in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (1.2.2)
Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (1.11.0)
Requirement not upgraded as not directly required: pyyaml in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (3.12)
Requirement not upgraded as not directly required: xarray>=0.10.1 in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (0.10.2)
Requirement not upgraded as not directly required: dask[array]>=0.17.1 in /usr/local/lib/python2.7/dist-packages (from satpy==0.9.0a2.dev0) (0.17.2)
Requirement not upgraded as not directly required: setuptools>=3.2 in /usr/local/lib/python2.7/dist-packages (from pyresample>=1.9.1->satpy==0.9.0a2.dev0) (39.1.0)
Requirement not upgraded as not directly required: pyproj>=1.9.5.1 in /usr/local/lib/python2.7/dist-packages (from pyresample>=1.9.1->satpy==0.9.0a2.dev0) (1.9.5.1)
Requirement not upgraded as not directly required: configobj in /usr/local/lib/python2.7/dist-packages (from pyresample>=1.9.1->satpy==0.9.0a2.dev0) (5.0.6)
Requirement not upgraded as not directly required: pandas>=0.18.0 in /usr/local/lib/python2.7/dist-packages (from xarray>=0.10.1->satpy==0.9.0a2.dev0) (0.22.0)
Requirement not upgraded as not directly required: toolz>=0.7.3; extra == "array" in /usr/local/lib/python2.7/dist-packages (from dask[array]>=0.17.1->satpy==0.9.0a2.dev0) (0.9.0)
Requirement not upgraded as not directly required: pytz>=2011k in /usr/lib/python2.7/dist-packages (from pandas>=0.18.0->xarray>=0.10.1->satpy==0.9.0a2.dev0) (2012rc0)
Requirement not upgraded as not directly required: python-dateutil in /usr/lib/python2.7/dist-packages (from pandas>=0.18.0->xarray>=0.10.1->satpy==0.9.0a2.dev0) (1.5)
matplotlib 1.3.1 requires tornado, which is not installed.
Installing collected packages: satpy
  Found existing installation: satpy 0.8.1
    Uninstalling satpy-0.8.1:
      Successfully uninstalled satpy-0.8.1
  Running setup.py install for satpy ... done
Successfully installed satpy-0.9.0a2.dev0
/usr/local/lib/python2.7/dist-packages/pip/_vendor/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning

[AranJM]

I am currently using a rather old Python version (2.7.6). This is a somewhat old server running legacy software.

[mraspaud]

@AranJM ok, I understand. Try installing the latest alpha release with pip install satpy --pre ..., the goes reading wasn't fixe in satpy 0.8.4

[AranJM]

@mraspaud Thanks for the help, but still no luck unfortunately. I tried the --pre option, but not sure what to replace the ... with. I tried 0.9.0, 0.9.0a0, 0.9.0a1, v0.9.0a0 etc., but they all resulted in an error saying version not found. I also tried the develop branch. This gave me no error messages, but the dataset was not loaded in the resampled scene.

[mraspaud]

@AranJM The numpy error should be fixed now with the latest commit.

[AranJM]

Tested and works! Thanks