Add a blend method to create temporal RGB from MultiScene #2488

pnuu · 2023-05-22T10:26:26Z

In #1748 we've discussed composites with temporal dependency. This is one attempt on one of the ideas.

import glob
from satpy import MultiScene
from satpy import Scene
from satpy.multiscene import temporal_rgb

# Three sets of data in ascending time order (latest last)
fnames = []
fnames.append(glob.glob('/data/*202305100800*))
fnames.append(glob.glob('/data/*202305100830*))
fnames.append(glob.glob('/data/*202305100855*))

scenes = [Scene(filenames=files, reader='seviri_l1b_hrit') for files in fnames]
mscn = MultiScene(scenes)
mscn.load(['HRV'])
mscn2 = mscn.resample('euron1', resampler='gradient_search')
blended = mscn2.blend(blend_function=temporal_rgb)
blended.save_datasets(base_dir='/tmp')

Closes #xxxx
Tests added
Fully documented

pnuu · 2023-05-22T10:31:45Z

And here is example from the above script, scaled down to smaller size:

And here's one cropped part in full 1 km resolution that better shows the effect.

The default enhancement for HRV (0...100) isn't the best for this use, so creating new composite with a dedicated enhancement will bring out the features better.

gerritholl · 2023-05-22T10:58:41Z

Nice start. But rather specific to the use case of a T-2/T-1/T-0 RGB. Would be interested in a discussion on how we can set up a more generic way of configuring this in YAML, where this would be just one specific case.

pnuu · 2023-05-22T12:12:48Z

Indeed, this is very specific to one of the cases, so didn't mark this to close the original issue.

I think the YAML variant would need to use glob() to find the files for the previous times. My thought was to:

add the functionality to Satpy for this T-2/T-1/T case
create a multi-temporal segment gatherer to pytroll-collectors to collect a range of time slots and publish a set with proper intervals (or approximates of them for polar data)
add a plugin to Trollflow2 to handle these messages and call MultiScene.blend(blend_function=temporal_rgb)

gerritholl · 2023-05-22T12:48:19Z

It looks like that if I have a MultiScene with more than three scenes, then all the scenes from the 4th onward get ignored?

pnuu · 2023-05-22T12:49:36Z

Yes, that's by design and tested. We have only three channels in any way 😉

gerritholl · 2023-05-22T12:58:28Z

I think the YAML variant would need to use glob() to find the files for the previous times

Unless we have some way for the MultiScene to look up a YAML configuration?

  temporal_rgb:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - name: HRV
      modifiers: [sunz_corrected]
      time: -2
    - name: HRV
      modifiers: [sunz_corrected]
      time: -1
    - name: HRV
      modifiers: [sunz_corrected]
      time: 0
    standard_name: temporal_rgb

This would be somehow found by mscn.load(...) or mscn.blend(...) (not sure).

Advantage: we can use modifiers and enhancements the same way we do for other composites — and we could replace GenericCompositor by something more fancy.

The weakness: I don't know how the "somehow" would work. A Scene couldn't load this composite, because a Scene has only one timestep.

In your example, how would modifiers and enhancements work?

pnuu · 2023-05-22T13:06:50Z

In your example, how would modifiers and enhancements work?

Simply by defining a (single-channel) composite for a given data:

  hrv:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: HRV
        modifiers: [sunz_corrected]
    standard_name: hrv

And to complement, an enhancement:

  hrv:
    standard_name: hrv
    operations:
    - name: stretch0
      method: *stretchfun
      kwargs:
        stretch: crude
        min_stretch: 0
        max_stretch: 100

And now instead of doing mscn.load(['HRV']) do mscn.load(['hrv']) to load the composite instead of the plain channel data. The results:

pnuu · 2023-05-22T13:14:46Z

I think the YAML variant would need to use glob() to find the files for the previous times

Unless we have some way for the MultiScene to look up a YAML configuration?

That's the point where we'd need to do some glob() or something to find the files needed for the previous time steps. By hand this is rather easy (like my script). There could also be a option for the MultiScene.group_files() to do this for the user 🤔 But for operations I think we need a new collector/gatherer to keep track of the last N scenes and then publish a suitable sets from them.

gerritholl · 2023-05-22T13:35:12Z

I think the YAML variant would need to use glob() to find the files for the previous times

Unless we have some way for the MultiScene to look up a YAML configuration?

That's the point where we'd need to do some glob() or something to find the files needed for the previous time steps.

Why? The MultiScene already has the scenes for each time step, provided by the user directly or (for operations) by a multi-timestep gatherer in pytroll-collectors.

satpy/multiscene/_multiscene.py

satpy/tests/multiscene_tests/test_blend.py

djhoese · 2023-05-22T14:12:39Z

I approve of this PR as-is, but understand that it doesn't fully satisfy all of the use cases that some people had in mind.

As you've pointed out the MultiScene doesn't really have a concept or understanding of what the Scenes are that are contained within it. It doesn't know if they're multiple orbits, multiple timesteps/repeat cycles, different satellites/sensors, etc. So there will always have to be some level of understanding by the user that "functionality X only works in these specific cases of the MultiScene". At least as currently designed.

One additional thing to consider is using the blend method with timeseries to produce a DataArray with a time dimension. The blend method will give you a Scene so you could technically then request blended_scn.load([some_composite]) where some_composite could error out if its inputs don't have a time dimension. Downsides would be that the user would have to know what inputs are required for the time-based composite so they can load them before the time-series blending. Another downside is that because the blended Scene was created with this time-based DataArray that product will be included in the "wishlist" and not as a dependency of a composite. It would have to be manually removed if the user no longer wanted it and/or any future calls to something like save_dataset would have to specify datasets=["some_composite"] to avoid using these time-based blended DataArrays.

Co-authored-by: David Hoese <david.hoese@ssec.wisc.edu>

pnuu · 2023-05-23T06:11:34Z

I think the YAML variant would need to use glob() to find the files for the previous times

Unless we have some way for the MultiScene to look up a YAML configuration?

That's the point where we'd need to do some glob() or something to find the files needed for the previous time steps.

Why? The MultiScene already has the scenes for each time step, provided by the user directly or (for operations) by a multi-timestep gatherer in pytroll-collectors.

True, but the composite loading happens at individual Scenes which have no information about other Scene objects, so the composite handling can't know about the other available time steps. I'll move over the discussion to #1748 for the additional stuff.

gerritholl · 2023-05-24T13:02:32Z

satpy/multiscene/_blend_funcs.py

+    compositor = GenericCompositor("temporal_composite")
+    composite = compositor((data_arrays[0], data_arrays[1], data_arrays[2]))


If there is time metadata available, this would probably fail with IncompatibleTimes, due to the check in composites.check_times.

Any idea which data has that so I could test? SEVIRI HRIT and AVHRR L1b AAPP worked.

I think in a lot of cases we started removing the time metadata in the readers because it was so inconsistent. I could be completely misremembering though.

codecov · 2023-05-25T07:33:22Z

Codecov Report

Merging #2488 (ec911db) into main (3b7a477) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #2488   +/-   ##
=======================================
  Coverage   94.83%   94.83%           
=======================================
  Files         337      337           
  Lines       49430    49461   +31     
=======================================
+ Hits        46875    46906   +31     
  Misses       2555     2555

Flag	Coverage Δ
behaviourtests	`4.43% <6.25%> (-0.01%)`	⬇️
unittests	`95.45% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
satpy/multiscene/_multiscene.py	`92.81% <ø> (ø)`
satpy/multiscene/__init__.py	`100.00% <100.00%> (ø)`
satpy/multiscene/_blend_funcs.py	`100.00% <100.00%> (ø)`
satpy/tests/multiscene_tests/test_blend.py	`100.00% <100.00%> (ø)`

coveralls · 2023-05-25T07:47:37Z

Pull Request Test Coverage Report for Build 5077181999

32 of 32 (100.0%) changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 95.402%

Totals
Change from base Build 5075221574:	0.003%
Covered Lines:	47019
Relevant Lines:	49285

💛 - Coveralls

gerritholl · 2023-05-25T13:03:38Z

satpy/multiscene/_blend_funcs.py

+    """
+    from satpy.composites import GenericCompositor
+
+    compositor = GenericCompositor("temporal_composite")


GenericCompositor removes units and calibration information, which we probably don't want if we produce an RGB from three times the same channel. Maybe GenericCompositor contains other (implicit) assumptions that make it not optimal for temporal composites.

Yes, good point. GenericCompositor is really "generic image compositor" so it assumes things no longer represent "real physical data" anymore.

Units don't make sense for the returned RGB, so that's fine.

Why would units not make sense if I combine three times HRV or three times 10.8 µm?

As this is creating an RGB, we are already implicitely renaming the time dimension with bands, so I get we already moved out of the scientific realm to move into the imaging one.

pnuu · 2023-10-06T05:28:52Z

Apparently the data should be from times T/T-1/T not T-2/T-1/T, but the code don't care so that's possible as long as the correct times are used.

mraspaud

LGTM!

Add a blend method to create temporal RGB from MultiScene

6d8eed7

pnuu added the component:multiscene label May 22, 2023

pnuu self-assigned this May 22, 2023

pnuu added 2 commits May 22, 2023 13:42

Remove extra parameters

72ccf1f

Fix using the attributes from the latest scene

3c48c53

Add tests for temporal RGB blending

ef355b4

pnuu marked this pull request as ready for review May 22, 2023 11:50

pnuu requested review from djhoese and mraspaud as code owners May 22, 2023 11:50

pnuu requested review from gerritholl and BENR0 May 22, 2023 11:50

Update MultiScene.blend() docstring

02b971c

djhoese reviewed May 22, 2023

View reviewed changes

satpy/multiscene/_multiscene.py Outdated Show resolved Hide resolved

djhoese reviewed May 22, 2023

View reviewed changes

satpy/tests/multiscene_tests/test_blend.py Outdated Show resolved Hide resolved

pnuu and others added 2 commits May 23, 2023 08:18

Update satpy/multiscene/_multiscene.py

652e325

Co-authored-by: David Hoese <david.hoese@ssec.wisc.edu>

Use explicit indices in test assertions

6c41a99

gerritholl mentioned this pull request May 23, 2023

Define and calculate composites with temporal dependencies #1748

Open

pnuu mentioned this pull request May 23, 2023

Add "multi-scene" collecting and publishing pytroll/pytroll-collectors#140

Open

gerritholl reviewed May 24, 2023

View reviewed changes

pnuu added 2 commits May 25, 2023 09:46

Merge branch 'main' into feature-multiscene-temporal-rgb-blend

1890bb2

Fix syntax error adding an item to a list

ec911db

gerritholl reviewed May 25, 2023

View reviewed changes

pnuu mentioned this pull request May 26, 2023

Add support for temporal composites #2495

Open

9 tasks

mraspaud approved these changes Oct 10, 2023

View reviewed changes

mraspaud added the enhancement code enhancements, features, improvements label Oct 10, 2023

mraspaud added this to the v0.44.0 milestone Oct 10, 2023

mraspaud merged commit ad543ef into pytroll:main Oct 11, 2023
16 checks passed

pnuu deleted the feature-multiscene-temporal-rgb-blend branch October 20, 2023 07:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a blend method to create temporal RGB from MultiScene #2488

Add a blend method to create temporal RGB from MultiScene #2488

pnuu commented May 22, 2023 •

edited

Loading

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023 •

edited by pnuu

Loading

djhoese commented May 22, 2023

pnuu commented May 23, 2023

gerritholl May 24, 2023

pnuu May 26, 2023

djhoese May 26, 2023

codecov bot commented May 25, 2023 •

edited

Loading

coveralls commented May 25, 2023

gerritholl May 25, 2023

djhoese May 25, 2023

pnuu May 26, 2023

gerritholl May 26, 2023

mraspaud Oct 10, 2023

pnuu commented Oct 6, 2023

mraspaud left a comment

		compositor = GenericCompositor("temporal_composite")
		composite = compositor((data_arrays[0], data_arrays[1], data_arrays[2]))

Add a blend method to create temporal RGB from MultiScene #2488

Add a blend method to create temporal RGB from MultiScene #2488

Conversation

pnuu commented May 22, 2023 • edited Loading

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023

pnuu commented May 22, 2023

pnuu commented May 22, 2023

gerritholl commented May 22, 2023 • edited by pnuu Loading

djhoese commented May 22, 2023

pnuu commented May 23, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented May 25, 2023 • edited Loading

Codecov Report

coveralls commented May 25, 2023

Pull Request Test Coverage Report for Build 5077181999

💛 - Coveralls

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pnuu commented Oct 6, 2023

mraspaud left a comment

Choose a reason for hiding this comment

pnuu commented May 22, 2023 •

edited

Loading

gerritholl commented May 22, 2023 •

edited by pnuu

Loading

codecov bot commented May 25, 2023 •

edited

Loading