Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

background compositor with colorized ir_clouds and static image problem #902

Closed
peters77 opened this issue Sep 12, 2019 · 2 comments · Fixed by #917
Closed

background compositor with colorized ir_clouds and static image problem #902

peters77 opened this issue Sep 12, 2019 · 2 comments · Fixed by #917
Milestone

Comments

@peters77
Copy link
Contributor

Describe the bug
using a via enhancement colorized ir_clouds composite doesn't work with a static background image merged with the background compositor and stops with a "KeyError: 'L'"

Expected behavior
Should work as expected: colorized Ir_clouds on top of a static background

Actual results

`[DEBUG: 2019-09-12 21:50:35 : satpy.readers.yaml_reader] Assigning to generic_image: ['/home/cpeters/satpy/backgrounds/BlackMarble_2016_3km_geo.tif']
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] Entering env context: <rasterio.env.Env object at 0x7f994a529550>
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] Starting outermost env
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] No GDAL environment exists
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] New GDAL environment <rasterio._env.GDALEnv object at 0x7f994a529630> created
[DEBUG: 2019-09-12 21:50:35 : rasterio._env] GDAL_DATA found in environment: '/home/cpeters/anaconda2/envs/pytroll/share/gdal'.
[DEBUG: 2019-09-12 21:50:35 : rasterio._env] PROJ_LIB found in environment: '/home/cpeters/anaconda2/envs/pytroll/share/proj'.
[DEBUG: 2019-09-12 21:50:35 : rasterio._env] Started GDALEnv <rasterio._env.GDALEnv object at 0x7f994a529630>.
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] Got a copy of environment <rasterio._env.GDALEnv object at 0x7f994a529630> options
[DEBUG: 2019-09-12 21:50:35 : rasterio.env] Entered env context: <rasterio.env.Env object at 0x7f994a529550>
[DEBUG: 2019-09-12 21:50:35 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2019-09-12 21:50:35 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2019-09-12 21:50:35 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2019-09-12 21:50:35 : rasterio._base] Dataset is started.

.....

/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/composites/init.py:354: UserWarning: satpy.composites.CompositeBase.check_areas is deprecated, use satpy.composites.CompositeBase.match_data_arrays instead
warnings.warn('satpy.composites.CompositeBase.check_areas is deprecated, use '
[DEBUG: 2019-09-12 21:52:56 : satpy.writers] Enhancement configuration options: [{'name': 'colorize_ngss', 'method': <function colorize at 0x7f99498c5ae8>, 'kwargs': {'palettes': [{'filename': '/home/cpeters/satpy/colormaps/ngss_colormap.npy', 'min_value': 153.15, 'max_value': 333.15}]}}]
[DEBUG: 2019-09-12 21:52:59 : satpy.writers] Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f99498c5510>, 'kwargs': {'stretch': 'crude', 'min_stretch': [0, 0, 0], 'max_stretch': [255, 255, 255]}}]
[DEBUG: 2019-09-12 21:52:59 : trollimage.xrimage] Applying stretch crude with parameters {'min_stretch': [0, 0, 0], 'max_stretch': [255, 255, 255]}
Traceback (most recent call last):
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'L'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "make_msg3_static_day.py", line 210, in
new_scene = global_scene.resample(area, resampler='bilinear', cache_dir='/tmp', fill_value=0)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 1103, in resample
keepables = new_scn.generate_composites()
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 884, in generate_composites
return self._read_composites(nodes)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 858, in _read_composites
self._generate_composite(item, keepables)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/scene.py", line 833, in _generate_composite
**self.attrs)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/satpy/composites/init.py", line 1469, in call
fg_band = foreground.sel(bands=band)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/dataarray.py", line 857, in sel
**indexers_kwargs)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/dataset.py", line 1722, in sel
self, indexers=indexers, method=method, tolerance=tolerance)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/coordinates.py", line 317, in remap_label_indexers
obj, v_indexers, method=method, tolerance=tolerance
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 252, in remap_label_indexers
dim, method, tolerance)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 181, in convert_label_indexer
indexer = get_loc(index, label.item(), method, tolerance)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/xarray/core/indexing.py", line 108, in get_loc
return index.get_loc(label, **kwargs)
File "/home/cpeters/anaconda2/envs/pytroll/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'L'`

Environment Info:

  • OS: Linux
  • Satpy Version: satpy-0.16.2.dev146
  • PyResample Version: pyresample-1.12.3+107
@djhoese
Copy link
Member

djhoese commented Oct 1, 2019

So I think I've figured out what's wrong here. The colorized_ir_clouds composite is configured to use the GenericCompositor which treats data as an image. Since it is a single band of data it defaults to image mode "L" (luminance). When the data is enhanced it is colorized with an RGB colormap, but trollimage sees the the mode attribute already set so it doesn't overwrite it. This results in something with bands [R, G, B] but a mode attribute of "L". Not great.

I think this could be considered a bug in trollimage's colorize method which should overwrite the mode attribute of the produced data. @pnuu @mraspaud Thoughts?

I don't think it is right for Satpy to assume anything about the mode if it is already set (could be CMYK or something). I think it is reasonable to assume that colorize knows what color mode it is applying to the data.

@djhoese
Copy link
Member

djhoese commented Oct 1, 2019

NOTE: Changing the compositor to @mraspaud's new SingleBandCompositor makes this work because it doesn't set an initial mode.

@mraspaud mraspaud added this to the v0.17.1 milestone Oct 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants