Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add 'day_night' flag to DayNightCompositor for day-only or night-only results #1816

Merged
merged 35 commits into from Sep 17, 2021
Merged

Add 'day_night' flag to DayNightCompositor for day-only or night-only results #1816

Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
9cce883
Update __init__.py
yukaribbba Sep 2, 2021
4a75db1
Update __init__.py
yukaribbba Sep 3, 2021
b3b7b35
Update __init__.py
yukaribbba Sep 3, 2021
2ceb218
Update __init__.py
yukaribbba Sep 9, 2021
80ed09a
Merge branch 'main' into feature-daynightcompositor
djhoese Sep 9, 2021
630baaf
tests
yukaribbba Sep 10, 2021
b294981
Revert "tests"
yukaribbba Sep 10, 2021
baf8501
Revert "Merge branch 'main' into feature-daynightcompositor"
yukaribbba Sep 10, 2021
6bcf7eb
Revert "Revert "Merge branch 'main' into feature-daynightcompositor""
yukaribbba Sep 10, 2021
82b84a6
update add_bands_test
yukaribbba Sep 10, 2021
a700799
update DayNightCompositor test
yukaribbba Sep 10, 2021
9bf4207
update DayNightCompositor test
yukaribbba Sep 10, 2021
ca5c3e9
new structure
yukaribbba Sep 11, 2021
f5c873b
Update __init__.py
yukaribbba Sep 11, 2021
73c2cdc
Update __init__.py
yukaribbba Sep 11, 2021
142e4a5
Update __init__.py
yukaribbba Sep 11, 2021
8d58657
Update __init__.py
yukaribbba Sep 11, 2021
926389e
Update __init__.py
yukaribbba Sep 11, 2021
70f43ee
Update __init__.py
yukaribbba Sep 11, 2021
360d876
Update test_composites.py
yukaribbba Sep 11, 2021
0f23b88
Update satpy/composites/__init__.py
yukaribbba Sep 13, 2021
1b14f7f
Update satpy/composites/__init__.py
djhoese Sep 13, 2021
a571014
Update __init__.py
yukaribbba Sep 13, 2021
4c00823
Update __init__.py
yukaribbba Sep 13, 2021
5c265b2
Update __init__.py
yukaribbba Sep 13, 2021
4f32c4c
Update __init__.py
yukaribbba Sep 13, 2021
8cdc9fa
Update __init__.py
yukaribbba Sep 13, 2021
cd55547
Update __init__.py
yukaribbba Sep 13, 2021
8d8377a
Update __init__.py
yukaribbba Sep 13, 2021
af2c6ee
Update __init__.py
yukaribbba Sep 14, 2021
787a6a9
Merge remote-tracking branch 'upstream/main' into feature-daynightcom…
yukaribbba Sep 16, 2021
0a75d4a
Update __init__.py
yukaribbba Sep 16, 2021
b59557f
Update composites.rst
yukaribbba Sep 17, 2021
9922f23
Update AUTHORS.md
yukaribbba Sep 17, 2021
872d95a
Update doc/source/composites.rst
djhoese Sep 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
126 changes: 99 additions & 27 deletions satpy/composites/__init__.py
View file
Open in desktop
Expand Up @@ -555,26 +555,29 @@ def _insert_palette_colors(channels, palette):
class DayNightCompositor(GenericCompositor):
"""A compositor that blends a day data with night data."""
yukaribbba marked this conversation as resolved.
Show resolved Hide resolved

def __init__(self, name, lim_low=85., lim_high=88., **kwargs):
def __init__(self, name, lim_low=85., lim_high=88., chose_day_night="day_and_night", **kwargs):
"""Collect custom configuration values.

Args:
lim_low (float): lower limit of Sun zenith angle for the
blending of the given channels
lim_high (float): upper limit of Sun zenith angle for the
blending of the given channels
chose_day_night (string): "day_and_night" means both day and night parts will be kept
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about day_night for the keyword argument (otherwise is should be choose_ not chose_)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the spell...

"only_day" means only day part will be kept
"only_night" means only night part will be kept

"""
self.lim_low = lim_low
self.lim_high = lim_high
self.chose_day_night = chose_day_night
super(DayNightCompositor, self).__init__(name, **kwargs)

def __call__(self, projectables, **kwargs):
"""Generate the composite."""
projectables = self.match_data_arrays(projectables)

day_data = projectables[0]
night_data = projectables[1]
data_1 = projectables[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a more descriptive name for this? I suppose "foreground" isn't really correct here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right. Whatever you choose, you need a 'foreground' image at least. So 'foreground' and 'background' are more suitable.


lim_low = np.cos(np.deg2rad(self.lim_low))
lim_high = np.cos(np.deg2rad(self.lim_high))
Expand All @@ -585,43 +588,99 @@ def __call__(self, projectables, **kwargs):
LOG.debug("Computing sun zenith angles.")
# Get chunking that matches the data
try:
chunks = day_data.sel(bands=day_data['bands'][0]).chunks
chunks = data_1.sel(bands=data_1['bands'][0]).chunks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github won't let me comment on it, but the coszen = line above uses projectables[2] which may or may not exist depending on the day/night mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This projectables[2] remains in the original one. I actually don't know what it's for...May be it's waiting for a solar zenith angle band/dataset from the satellite data?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 Oh duh. That is if the YAML file specifies the solar_zenith_angle dataset from the reader (ex. for viirs_sdr). Geostationary satellites don't typically have that provided by the file/reader so it isn't specified. You may want this to be projectables[2 if self.day_night == "day_night" else 1].

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😊OK It's late night here so I'll do these changes tomorrow.

except KeyError:
chunks = day_data.chunks
lons, lats = day_data.attrs["area"].get_lonlats(chunks=chunks)
coszen = xr.DataArray(cos_zen(day_data.attrs["start_time"],
chunks = data_1.chunks
lons, lats = data_1.attrs["area"].get_lonlats(chunks=chunks)
coszen = xr.DataArray(cos_zen(data_1.attrs["start_time"],
lons, lats),
dims=['y', 'x'],
coords=[day_data['y'], day_data['x']])
coords=[data_1['y'], data_1['x']])
# Calculate blending weights
coszen -= np.min((lim_high, lim_low))
coszen /= np.abs(lim_low - lim_high)
coszen = coszen.clip(0, 1)

# Apply enhancements to get images
day_data = enhance2dataset(day_data)
night_data = enhance2dataset(night_data)
if self.chose_day_night == "day_and_night":
# This means both day and night part will be kept
data_2 = projectables[1]
day_data = data_1
night_data = data_2

# Adjust bands so that they match
# L/RGB -> RGB/RGB
# LA/RGB -> RGBA/RGBA
# RGB/RGBA -> RGBA/RGBA
day_data = add_bands(day_data, night_data['bands'])
night_data = add_bands(night_data, day_data['bands'])
# Apply enhancements to get images
day_data = enhance2dataset(day_data)
night_data = enhance2dataset(night_data)

# Replace missing channel data with zeros
day_data = zero_missing_data(day_data, night_data)
night_data = zero_missing_data(night_data, day_data)
# Adjust bands so that they match
# L/RGB -> RGB/RGB
# LA/RGB -> RGBA/RGBA
# RGB/RGBA -> RGBA/RGBA
day_data = add_bands(day_data, night_data['bands'])
night_data = add_bands(night_data, day_data['bands'])

# Get merged metadata
attrs = combine_metadata(day_data, night_data)
# Replace missing channel data with zeros
day_data = zero_missing_data(day_data, night_data)
night_data = zero_missing_data(night_data, day_data)

# Get merged metadata
attrs = combine_metadata(day_data, night_data)

# Blend the two images together
data = (1 - coszen) * night_data + coszen * day_data
data.attrs = attrs

# Split to separate bands so the mode is correct
data = [data.sel(bands=b) for b in data['bands']]

elif self.chose_day_night == "only_day":
# This means only day part will be kept
day_data = data_1

# Apply enhancements to get images
day_data = enhance2dataset(day_data)

# Add alpha band
# L -> LA
# RGB -> RGBA
day_data = add_bands(day_data, day_data['bands'])

# Replace missing channel data with zeros
day_data = zero_missing_data(day_data, day_data)

# Get merged metadata
attrs = combine_metadata(day_data, day_data)

# Set night part to 0 and get the maskout image
data = (1 - coszen) * 0 + coszen * day_data
data.attrs = attrs

# Split to separate bands so the mode is correct
data = [data.sel(bands=b) for b in data['bands']]

# Blend the two images together
data = (1 - coszen) * night_data + coszen * day_data
data.attrs = attrs
elif self.chose_day_night == "only_night":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the day/night only modes be combined with if statements for the things that are different? You could also ignore the 0 parameters in the logic so that extra computation isn't done when it isn't needed. Oh you might be able to combine all three day/night modes...

day_data = projectables[0] if "day" in self.day_night else None
night_data = projectables[0] if self.day_night == "night_only" else None
night_data = projectables[1] if self.day_night == "day_night" else night_data

day_portion = coszen * (day_data if day_data is not None else 0)
night_portion = (1 - coszen) * (night_data if night_data is not None else 0)
data = night_portion + day_portion

You'd need some more logic for handling the add_bands and zero_missing_data stuff, but it may actually make the code a lot smaller. Maybe not clearer though... 😨

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I also think there's something need to be improved in the logic structure. I'll go look into this.

# This means only night part will be kept
night_data = data_1

# Split to separate bands so the mode is correct
data = [data.sel(bands=b) for b in data['bands']]
# Apply enhancements to get images
night_data = enhance2dataset(night_data)

# Add alpha band
# L -> LA
# RGB -> RGBA
night_data = add_bands(night_data, night_data['bands'])

# Replace missing channel data with zeros
night_data = zero_missing_data(night_data, night_data)

# Get merged metadata
attrs = combine_metadata(night_data, night_data)

# Set night part to 0 and get the maskout image
data = (1 - coszen) * night_data + coszen * 0
data.attrs = attrs

# Split to separate bands so the mode is correct
data = [data.sel(bands=b) for b in data['bands']]

return super(DayNightCompositor, self).__call__(data, **kwargs)

Expand Down Expand Up @@ -695,6 +754,19 @@ def add_bands(data, bands):
new_data = xr.concat(new_data, dim='bands')
new_data.attrs['mode'] = data.attrs['mode'] + 'A'
data = new_data
elif 'A' not in data['bands'].data:
new_data = [data.sel(bands=band) for band in data['bands'].data]
# Create alpha band based on a copy of the first "real" band
alpha = new_data[0].copy()
alpha.data = da.ones((data.sizes['y'],
data.sizes['x']),
chunks=new_data[0].chunks)
# Rename band to indicate it's alpha
alpha['bands'] = 'A'
new_data.append(alpha)
new_data = xr.concat(new_data, dim='bands')
new_data.attrs['mode'] = data.attrs['mode'] + 'A'
data = new_data

return data

Expand Down