Creating composites post-load

[pnuu]

I'd like to create composites after resampling, not at load time. This would save both memory and (CPU) time, especially in the case of large native resolution data resampled to a much smaller domain. There's always the option to use mpop.image.GeoImage or trollimage.Image, but I think it'd be cleaner to use the already existing built-in mechanics.

[pnuu]

After some thought (10 seconds), my suggestion is that the composite names could still be given to glbl.load() so that the required channels are loaded, but the composites are generated only when said composite is being saved (or lcl.show(dataset_name)'d).

[djhoese]

This should be possible by specifying scn.load(..., compute=False).

It could be possible to automate this decision by taking in to account some things from the user:

1. Number and size of target areas
2. Number and size of composite inputs (size of input arrays, number of inputs for each composite, L or RGB composite)

IIRC we talked about this briefly at the Norrköping workshop. Sometimes you are generating a ton of composites from 4 or 5 bands so it makes sense to wait until after resampling. Sometimes you are generating 2 composites from 6-8 bands so it makes sense to compute them before resampling.

[pnuu]

Oh, ok, compute=False does what I was thinking in the later post.

Would it be possible to expose also the built-in compositors to the resampled Scene? So that, assuming the required channels are loaded, it would be possible to do something like composite = scn.build_composite('overview').

[djhoese]

Would it be possible to expose also the built-in compositors to the resampled Scene? So that, assuming the required channels are loaded, it would be possible to do something like composite = scn.build_composite('overview').

Yes, but I'm not looking forward to doing it 😄

[djhoese]

Thinking about this more, there may be a rare case where a compositor is specific to a sensor or depends on a polar instruments swath size to do it's calculations. Or they may assume that they have a swath so their calculations don't take in to account "extra" empty pixels from the gridding process.

[djhoese]

The part of this issue related to the compute=False keyword is now named generate=False and does the same tasks. Given our use of dask arrays and lazy loading/computing data I'm not sure the use case for this is as useful as reduce_data in the resampling step will attempt to minimize the dask array chunks that get processed.

@pnuu If you don't think this is a good enough answer feel free to reopen.