Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. #301

sjoro · 2018-05-18T13:13:45Z

Fix a bug in hrit_msg and native_msg header-definitions. Factorize header-definitions between hrit_msg and native_msg. Add image trailer reading for native_msg. Clean-up code.

Tests added
Tests passed
Passes git diff origin/master **/*py | flake8 --diff

…finition, all bytes are correct now. Add trailer reading to native_msg. Code clean-up.

coveralls · 2018-05-18T13:25:13Z

Coverage increased (+0.5%) to 66.345% when pulling a424fc6 on sjoro:master into 89217cb on pytroll:master.

codecov · 2018-05-18T13:32:55Z

Codecov Report

Merging #301 into master will increase coverage by 0.47%.
The diff coverage is 39.66%.

@@            Coverage Diff             @@
##           master     #301      +/-   ##
==========================================
+ Coverage   65.86%   66.34%   +0.47%     
==========================================
  Files         118      119       +1     
  Lines       14336    14414      +78     
==========================================
+ Hits         9443     9563     +120     
+ Misses       4893     4851      -42

Impacted Files	Coverage Δ
satpy/readers/msg_base.py	`96.96% <ø> (ø)`	⬆️
satpy/tests/reader_tests/test_msg_base.py	`84.61% <ø> (ø)`	⬆️
satpy/tests/reader_tests/test_eum_base.py	`0% <0%> (ø)`
satpy/tests/reader_tests/test_native_msg.py	`85.71% <100%> (-0.5%)`	⬇️
satpy/tests/reader_tests/test_hrit_goes.py	`75.32% <100%> (ø)`	⬆️
satpy/readers/native_msg_hdr.py	`100% <100%> (+71.02%)`	⬆️
satpy/readers/hrit_base.py	`81.41% <100%> (+5.41%)`	⬆️
satpy/readers/hrit_goes.py	`50.68% <100%> (ø)`	⬆️
satpy/tests/reader_tests/test_hrit_base.py	`93.84% <100%> (-1.16%)`	⬇️
satpy/readers/native_msg.py	`15.84% <17.91%> (-2.72%)`	⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89217cb...a424fc6. Read the comment docs.

…um_base.py for hosting common reader utilities for EUMETSAT data.

stickler-ci · 2018-05-25T12:26:41Z

satpy/readers/hrit_msg.py

@@ -38,16 +38,20 @@
 import numpy as np

 from pyresample import geometry
+
+from satpy.readers.eum_base import (make_time_cds, time_cds_short,


F401 'satpy.readers.eum_base.make_time_cds' imported but unused

stickler-ci · 2018-05-25T12:26:41Z

satpy/readers/native_msg.py

-
-class CalibrationError(Exception):
-    pass
+from satpy.readers.eum_base import make_time_cds, recarray2dict


F401 'satpy.readers.eum_base.make_time_cds' imported but unused

stickler-ci · 2018-05-25T12:26:42Z

satpy/readers/native_msg.py

-class CalibrationError(Exception):
-    pass
+from satpy.readers.eum_base import make_time_cds, recarray2dict
+from satpy.readers.msg_base import (SEVIRICalibrationHandler,


F401 'satpy.readers.msg_base.get_cds_time' imported but unused

stickler-ci · 2018-05-25T12:26:42Z

satpy/readers/native_msg.py


        data15hd = self.header['15_DATA_HEADER']
        sec15hd = self.header['15_SECONDARY_PRODUCT_HEADER']

+        utc = data15hd['ImageAcquisition']['PlannedAcquisitionTime']


F841 local variable 'utc' is assigned to but never used

stickler-ci · 2018-05-25T12:26:42Z

satpy/readers/native_msg.py

-
+        west = int(sec15hd['WestColumnSelectedRectangle']['Value'])
+        east = int(sec15hd['EastColumnSelectedRectangle']['Value'])
+        north = int(sec15hd["NorthLineSelectedRectangle"]['Value'])


F841 local variable 'north' is assigned to but never used

stickler-ci · 2018-05-25T12:26:42Z

satpy/readers/native_msg.py

+        west = int(sec15hd['WestColumnSelectedRectangle']['Value'])
+        east = int(sec15hd['EastColumnSelectedRectangle']['Value'])
+        north = int(sec15hd["NorthLineSelectedRectangle"]['Value'])
+        south = int(sec15hd["SouthLineSelectedRectangle"]['Value'])


F841 local variable 'south' is assigned to but never used

stickler-ci · 2018-05-25T12:26:42Z

satpy/tests/reader_tests/test_hrit_msg.py

-                                    make_time_cds_expanded)
+from satpy.readers.hrit_msg import HRITMSGPrologueFileHandler
+# make_time_cds_expanded)
+from satpy.readers.eum_base import make_time_cds


F401 'satpy.readers.eum_base.make_time_cds' imported but unused

sjoro · 2018-05-25T12:44:50Z

[ x] Passes test_hrit_base, test_msg_base, test_eum_base, test_native_msg

Fails on test_hrit_goes, test_hrit_msg, need help here to fix the unittests.

mraspaud

Good job !

mraspaud · 2018-05-25T13:36:42Z

satpy/readers/hrit_goes.py

@@ -49,6 +48,30 @@ class CalibrationError(Exception):
 logger = logging.getLogger('hrit_goes')


+def recarray2dict(arr):


This function should be removed, right ?

mraspaud · 2018-05-25T13:38:17Z

satpy/readers/hrit_msg.py

@@ -451,8 +116,6 @@ def __init__(self, filename, filename_info, filetype_info):
                                                         (msg_hdr_map,
                                                          msg_variable_length_headers,
                                                          msg_text_headers))
-
-        self.prologue = {}


instance attributes should be initialized in init

…m_base. Remove unittest for hrit_msg. Add unittest for recarray2dict in eum_base. Clean-up code.

mraspaud

LGTM

sjoro added 3 commits May 18, 2018 07:27

Factorize HRIT headers using native_msg_hdr. Fix native_msg header-de…

9e2f440

…finition, all bytes are correct now. Add trailer reading to native_msg. Code clean-up.

Unify naming in time_cds-records for millicesonds

744a0b5

Move prologue and epilogue records under read-methods

4c5a3ed

mraspaud assigned sjoro May 21, 2018

mraspaud added enhancement code enhancements, features, improvements component:readers labels May 21, 2018

mraspaud changed the title ~~Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition.~~ [WIP] Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. May 21, 2018

sjoro added 3 commits May 23, 2018 06:08

Merge branch 'master' of https://github.com/pytroll/satpy

ae0fb66

Factorize SEVIRI HRIT and NATIVE readers. Reorganize unittests. Add e…

1d3aff3

…um_base.py for hosting common reader utilities for EUMETSAT data.

Clean-up unused imports and variables.

334bb95

stickler-ci reviewed May 25, 2018

View reviewed changes

mraspaud reviewed May 25, 2018

View reviewed changes

sjoro and others added 4 commits May 28, 2018 05:53

Add units for counts in hrit_msg.yaml

b19ba1e

Initialize instance attributes in init. Factorize hrit_goes to use eu…

e1f1d57

…m_base. Remove unittest for hrit_msg. Add unittest for recarray2dict in eum_base. Clean-up code.

Fix native_msg-reader to use counts as calibration type.

dff101d

Fix test_hrit_goes prologue test

a424fc6

mraspaud approved these changes May 29, 2018

View reviewed changes

mraspaud changed the title ~~[WIP] Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition.~~ Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. May 29, 2018

mraspaud merged commit 93826db into pytroll:master May 29, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. #301

Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. #301

sjoro commented May 18, 2018 •

edited by mraspaud

Loading

coveralls commented May 18, 2018 •

edited

Loading

codecov bot commented May 18, 2018 •

edited

Loading

stickler-ci May 25, 2018

stickler-ci May 25, 2018

stickler-ci May 25, 2018

stickler-ci May 25, 2018

stickler-ci May 25, 2018

stickler-ci May 25, 2018

stickler-ci May 25, 2018

sjoro commented May 25, 2018 •

edited

Loading

mraspaud left a comment

mraspaud May 25, 2018

mraspaud May 25, 2018

mraspaud left a comment

		@@ -49,6 +48,30 @@ class CalibrationError(Exception):
		logger = logging.getLogger('hrit_goes')


		def recarray2dict(arr):

Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. #301

Factorize header definitions between hrit_msg and native_msg. Fix a bug in header definition. #301

Conversation

sjoro commented May 18, 2018 • edited by mraspaud Loading

coveralls commented May 18, 2018 • edited Loading

codecov bot commented May 18, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjoro commented May 25, 2018 • edited Loading

mraspaud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

sjoro commented May 18, 2018 •

edited by mraspaud

Loading

coveralls commented May 18, 2018 •

edited

Loading

codecov bot commented May 18, 2018 •

edited

Loading

sjoro commented May 25, 2018 •

edited

Loading