Transverse Mercator section added in cf writer #920

bornagain1981 · 2019-10-03T08:33:48Z

Tests added and test suite added to parent suite
Tests passed
Passes flake8 satpy
Fully documented
Add your name to AUTHORS.md if not there already

Transverse Mercator section added in cf writer

coveralls · 2019-10-03T08:51:46Z

Coverage increased (+0.5%) to 87.366% when pulling 26bdcf4 on bornagain1981:nc_mod into 1fbfda5 on pytroll:master.

mraspaud

@bornagain1981 Thank you very much for adding the transverse mercator to the cf writer! The modifications look good, good job. Just a couple of things:

could you fix the style error flagged by Stickler CI ?
and could you add a simple test like this one : https://github.com/pytroll/satpy/blob/master/satpy/tests/writer_tests/test_cf.py#L680-L715 ?

codecov · 2019-10-06T11:06:05Z

Codecov Report

Merging #920 into master will increase coverage by 0.5%.
The diff coverage is 92.59%.

@@            Coverage Diff            @@
##           master     #920     +/-   ##
=========================================
+ Coverage   86.86%   87.37%   +0.5%     
=========================================
  Files         181      183      +2     
  Lines       27461    28179    +718     
=========================================
+ Hits        23853    24620    +767     
+ Misses       3608     3559     -49

Impacted Files	Coverage Δ
satpy/tests/writer_tests/test_cf.py	`98.49% <100%> (+0.12%)`	⬆️
satpy/writers/cf_writer.py	`90.67% <75%> (-0.6%)`	⬇️
satpy/tests/utils.py	`93.87% <0%> (-3.21%)`	⬇️
satpy/readers/ahi_hsd.py	`96.58% <0%> (-0.31%)`	⬇️
satpy/readers/avhrr_l1b_gaclac.py	`94.59% <0%> (-0.26%)`	⬇️
satpy/tests/reader_tests/test_abi_l2_nc.py	`93.84% <0%> (-0.19%)`	⬇️
satpy/readers/goes_imager_hrit.py	`84.86% <0%> (ø)`	⬆️
satpy/readers/_geos_area.py	`100% <0%> (ø)`	⬆️
satpy/readers/nwcsaf_msg2013_hdf5.py	`96.1% <0%> (ø)`	⬆️
... and 50 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1fbfda5...26bdcf4. Read the comment docs.

bornagain1981 · 2019-10-06T14:43:18Z

@mraspaud during test : 2 errors and some warnings (i receive the same during the test cf writer in branch master)

(pytroll) bornagain@bornlaptop:~/satpy/satpy$ python3 tests/writer_tests/test_cf.py
../home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/pyresample/geometry.py:1604: DeprecationWarning: 'get_lonlats_dask' is deprecated, please use 'get_lonlats' with the 'chunks' keyword argument specified.
"'get_lonlats' with the 'chunks' keyword argument specified.", DeprecationWarning)
/home/bornagain/anaconda3/envs/pytroll/lib/python3.7/unittest/case.py:1143: DeprecationWarning: assertDictContainsSubset is deprecated
DeprecationWarning)
../home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/xarray/conventions.py:393: UserWarning: Variable 'time' has datetime type and a bounds variable but time.encoding does not have units specified. The units encodings for 'time' and 'time_bnds' will be determined independently and may not be equal, counter to CF-conventions. If this is a concern, specify a units encoding for 'time' before writing to a file.
UserWarning)
./home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/xarray/conventions.py:393: UserWarning: Variable 'time' has datetime type and a bounds variable but time.encoding does not have units specified. The units encodings for 'time' and 'time_bnds' will be determined independently and may not be equal, counter to CF-conventions. If this is a concern, specify a units encoding for 'time' before writing to a file.
UserWarning)
./home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/xarray/conventions.py:393: UserWarning: Variable 'time' has datetime type and a bounds variable but time.encoding does not have units specified. The units encodings for 'time' and 'time_bnds' will be determined independently and may not be equal, counter to CF-conventions. If this is a concern, specify a units encoding for 'time' before writing to a file.
UserWarning)
..FFNo time dimension in datasets, skipping time bounds creation.
./home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/satpy/writers/cf_writer.py:368: UserWarning: Cannot pretty-format "acq_time" coordinates because they are not unique among the given datasets
'given datasets'.format(coord_name))
No time dimension in datasets of group visir, skipping time bounds creation.
No time dimension in datasets of group hrv, skipping time bounds creation.
.No time dimension in datasets, skipping time bounds creation.
....No time dimension in datasets, skipping time bounds creation.
.No time dimension in datasets, skipping time bounds creation.
../home/bornagain/anaconda3/envs/pytroll/lib/python3.7/site-packages/xarray/conventions.py:393: UserWarning: Variable 'time' has datetime type and a bounds variable but time.encoding does not have units specified. The units encodings for 'time' and 'time_bnds' will be determined independently and may not be equal, counter to CF-conventions. If this is a concern, specify a units encoding for 'time' before writing to a file.
UserWarning)

FAIL: test_da2cf (main.TestCFWriter)
Test the conversion of a DataArray to a CF-compatible DataArray.

Traceback (most recent call last):
File "tests/writer_tests/test_cf.py", line 483, in test_da2cf
self.assertDictWithArraysEqual(res.attrs, attrs_expected)
File "tests/writer_tests/test_cf.py", line 426, in assertDictWithArraysEqual
self.assertEqual(val1, val2)
AssertionError: 'True' != True

FAIL: test_encode_attrs_nc (main.TestCFWriter)
Test attributes encoding.

Traceback (most recent call last):
File "tests/writer_tests/test_cf.py", line 440, in test_encode_attrs_nc
self.assertDictWithArraysEqual(expected, encoded)
File "tests/writer_tests/test_cf.py", line 428, in assertDictWithArraysEqual
self.assertTrue(isinstance(val2, np.generic))
AssertionError: False is not true

Ran 20 tests in 0.402s

FAILED (failures=2)
(pytroll) bornagain@bornlaptop:~/satpy/satpy$

mraspaud · 2019-10-08T13:06:33Z

@bornagain1981 ok, do you still have the errors with master ? the warnings are ok I think

bornagain1981 · 2019-10-29T16:18:57Z

@bornagain1981 ok, do you still have the errors with master ? the warnings are ok I think

yes, same error using branch-master, different computers or launching test in another conda environment

bornagain1981 · 2019-11-27T11:14:35Z

Test_cf is ok and passed also the other test:

"""Test the conversion from pyresample area object to CF grid mapping."""
import xarray as xr
import pyresample.geometry
from satpy.writers.cf_writer import area2gridmapping

def assertEqual(var1, var2):
    if var1 == var2:
        return True
    else:
        return False

ds_base = xr.DataArray(data=[[1, 2], [3, 4]], dims=('y', 'x'), coords={'y': [1, 2], 'x': [3, 4]},
                       attrs={'name': 'var1'})
# a) Projection has a corresponding CF representation (e.g. geos)
lat_0= 36.5
lon_0= 15.0
lat_ts= 36.5

tmerc = pyresample.geometry.AreaDefinition(
    area_id='tmerc',
    description='tmerc',
    proj_id='tmerc',
    projection={'proj': 'tmerc', 'ellps': 'WGS84', 'lat_0': 36.5, 'lon_0': 15.0,'lat_ts': 36.5},
    width=2, height=2,
    area_extent=[-1, -1, 1, 1])

tmerc_expected = xr.DataArray(data=0,
                             attrs={'azimuth_of_central_line':'alpha',
                                    'latitude_of_projection_origin':lat_0,
                                    'longitude_of_projection_origin':lon_0,
                                    'latitude_of_meridian_ts':lat_ts,
                                    'grid_mapping_name':'transverse_mercator',
                                    'reference_ellipsoid_name':('ellps', 'WGS84'),
                                    'prime_meridian_name':('pm', 'Greenwich'),
                                    'horizontal_datum_name':('datum', 'unknown'),
                                    'geographic_crs_name':'unknown',
                                    'false_easting':0.,
                                    'false_northing':0.,
                                    'name': 'tmerc'})


ds = ds_base.copy()
ds.attrs['area'] = tmerc
res, grid_mapping = area2gridmapping(ds)

assertEqual(grid_mapping, tmerc_expected)

True

assertEqual(res.attrs['grid_mapping'], 'tmerc')

True

mraspaud · 2019-12-11T23:33:35Z

Nice work @bornagain1981 ! I think you just need to add your name to the AUTHORS.md file, the rest looks fine

bornagain1981 added 2 commits October 3, 2019 10:14

Transverse Mercator section added in cf writer

ef77c48

Merge pull request #1 from bornagain1981/Dario_nc_mod

103fecf

Transverse Mercator section added in cf writer

bornagain1981 requested a review from mraspaud as a code owner October 3, 2019 08:33

mraspaud requested changes Oct 3, 2019

View reviewed changes

Update cf_writer.py

e4045de

mraspaud assigned bornagain1981 Oct 8, 2019

mraspaud added component:writers enhancement code enhancements, features, improvements labels Oct 8, 2019

bornagain1981 added this to In progress in PCW Copenhagen 2019 Nov 21, 2019

Merge branch 'master' into nc_mod

11d96a4

last commit

6fc0543

bornagain1981 requested a review from djhoese as a code owner December 9, 2019 20:28

bornagain1981 added 6 commits December 10, 2019 16:59

Added Transverse Mercator Test

fae000b

last

3b7802d

last commit

227756a

last commit

eaf9440

commit

6346219

commit

a45a8fc

added my name to Authors.md

26bdcf4

mraspaud approved these changes Dec 12, 2019

View reviewed changes

mraspaud merged commit 3ba7b3d into pytroll:master Dec 12, 2019

PCW Copenhagen 2019 automation moved this from In progress to Done Dec 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transverse Mercator section added in cf writer #920

Transverse Mercator section added in cf writer #920

bornagain1981 commented Oct 3, 2019 •

edited

coveralls commented Oct 3, 2019 •

edited

mraspaud left a comment

codecov bot commented Oct 6, 2019 •

edited

bornagain1981 commented Oct 6, 2019

mraspaud commented Oct 8, 2019

bornagain1981 commented Oct 29, 2019

bornagain1981 commented Nov 27, 2019

mraspaud commented Dec 11, 2019

Transverse Mercator section added in cf writer #920

Transverse Mercator section added in cf writer #920

Conversation

bornagain1981 commented Oct 3, 2019 • edited

coveralls commented Oct 3, 2019 • edited

mraspaud left a comment

Choose a reason for hiding this comment

codecov bot commented Oct 6, 2019 • edited

Codecov Report

bornagain1981 commented Oct 6, 2019

mraspaud commented Oct 8, 2019

bornagain1981 commented Oct 29, 2019

bornagain1981 commented Nov 27, 2019

mraspaud commented Dec 11, 2019

bornagain1981 commented Oct 3, 2019 •

edited

coveralls commented Oct 3, 2019 •

edited

codecov bot commented Oct 6, 2019 •

edited