Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix BackgroundCompositor not retaining input metadata #917

Merged
merged 2 commits into from Oct 2, 2019

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Sep 30, 2019

While working on #854 I noticed that sensor information wasn't being handled the way I wanted by the BackgroundCompositor. I then discovered that none of the metadata is being handled. This PR attempts to fix that and cleans up a few other things that flake8/pre-commit were complaining about.

This includes removing the use of some deprecated functions.

This might fix #902 (FYI @peters77)

  • Tests added and test suite added to parent suite
  • Tests passed
  • Passes flake8 satpy
  • Fully documented

@peters77
Copy link
Contributor

peters77 commented Sep 30, 2019

Thanks for the info. #902 seems not to be fixed by your modification, here my output:

  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'L'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "make_msg3_germ_day.py", line 207, in <module>
    new_scene = global_scene.resample(area, radius_of_influence = 20000, nprocs=4)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 1103, in resample
    keepables = new_scn.generate_composites()
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 884, in generate_composites
    return self._read_composites(nodes)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 858, in _read_composites
    self._generate_composite(item, keepables)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 833, in _generate_composite
    **self.attrs)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/composites/__init__.py", line 1520, in __call__
    fg_band = foreground.sel(bands=band)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/dataarray.py", line 857, in sel
    **indexers_kwargs)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/dataset.py", line 1722, in sel
    self, indexers=indexers, method=method, tolerance=tolerance)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/coordinates.py", line 317, in remap_label_indexers
    obj, v_indexers, method=method, tolerance=tolerance
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 252, in remap_label_indexers
    dim, method, tolerance)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 181, in convert_label_indexer
    indexer = get_loc(index, label.item(), method, tolerance)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 108, in get_loc
    return index.get_loc(label, **kwargs)
  File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'L' ```

Copy link
Member

@pnuu pnuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Unfortunately can't test within the couple next days.

satpy/composites/__init__.py Outdated Show resolved Hide resolved
@pnuu
Copy link
Member

pnuu commented Oct 1, 2019

Should've first checked the test results, but the fix seems to be pretty trivial :-)

@djhoese djhoese requested a review from adybbroe as a code owner October 1, 2019 15:21
@djhoese
Copy link
Member Author

djhoese commented Oct 1, 2019

@peters77 If you try this branch again it should fix your issue.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) to 85.348% when pulling 5e92278 on djhoese:bugfix-background-compositor into d09edbc on pytroll:master.

@codecov
Copy link

codecov bot commented Oct 1, 2019

Codecov Report

Merging #917 into master will decrease coverage by 0.01%.
The diff coverage is 88.67%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #917      +/-   ##
==========================================
- Coverage   85.36%   85.35%   -0.02%     
==========================================
  Files         174      174              
  Lines       26023    26037      +14     
==========================================
+ Hits        22215    22223       +8     
- Misses       3808     3814       +6
Impacted Files Coverage Δ
satpy/tests/compositor_tests/__init__.py 99.54% <100%> (ø) ⬆️
satpy/composites/__init__.py 75.09% <72.72%> (-0.26%) ⬇️
satpy/scene.py 90.47% <0%> (-0.18%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d09edbc...5e92278. Read the comment docs.

@peters77
Copy link
Contributor

peters77 commented Oct 1, 2019

@djhoese issue fixed if I use this branch. 👍 Now if #830 is fixed I can use a static day background image too.
MSG-3-test-raw-4

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks for taking the time to sort things out!

@djhoese djhoese merged commit 95b0aea into pytroll:master Oct 2, 2019
@djhoese djhoese deleted the bugfix-background-compositor branch October 2, 2019 11:28
@mraspaud mraspaud added this to the v0.17.1 milestone Oct 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

background compositor with colorized ir_clouds and static image problem
5 participants