Federation of Secondary Services

[soxofaan]

In #78 we started with initial support for secondary services in the aggregator. The following question(s) came up during initial experimentation.

With collections and processes there has been a strong push for harmonization across back-ends, which allows offering "unified" entities at the level of the aggregator, e.g. a collection SENTINEL2_L2A that is supported by both VITO and SentinelHub, and which will be dispatched by the aggregator based on requested spatial/temporal extent.

Secondary services didn't receive same level of attention, until recently. There is a "configuration" aspect when creating a service, which is largely back-end specific, or at least, there hasn't been an effort to try to harmonize this. This makes it less straightforward to support at the federation/aggregator level.

Question is now: how are we going to handle this (in the short term)?

1. push for harmonization for secondary service configuration (e.g. in federation extension), like we do for collections and processes, and aim for "unified" secondary services, in the sense that the user does not have to be aware of which upstream back-end is handling the secondary service.
2. There is no unification and user is explicitly exposed to different secondary service options on each upstream back-end, each with original configuration options. For example: the service types are prefixed with backend id: sentinelhub-XYZ, vito-WMTS, ... This means that the user must understand federation aspects like "if I use this collection (provided by upstream backend X), I can only use this secondary service type X-...".

The first option is probably the ideal option (in long term), but pretty ambitious in terms of harmonization/specification effort.
So at the moment I'm more included towards option 2, which is uglier, but a lot easier to implement, and it allows us to experiment more with the feature before committing to something under option 1.

[soxofaan]

@m-mohr @jdries @dthiex any opinions here?

[m-mohr]

The configuration and the process_parameters are something to be discussed. Ideally, this would be standardized in the Federation contract, but that can only be done once we have a second implementation, right? Are there any planned?

[soxofaan]

but that can only be done once we have a second implementation, right? Are there any planned?

We had an initial implementation in VITO backend, but it grew defunct due to lack of interest/maintenance. We don't have short term plans to fix that I think, right @jdries ?

[m-mohr]

And it was a WMTS, not an XYZ as for SH, right? So these would not be "combined"/unified...

[soxofaan]

WMTS indeed, so for now we can workaround/ignore the problem because there is no conflict yet

[dthiex]

WMTS indeed, so for now we can workaround/ignore the problem because there is no conflict yet

I agree and also think for now (where only one of the backends supports this we can go with the simple solution). Once a 2nd backend wants to offer this it will be easier to discuss on a concrete example and then also test it.

This means that the user must understand federation aspects like "if I use this collection (provided by upstream backend X), I can only use this secondary service type X-...".

Yes but I think if the backend provides a meaningful error message in such a case it shouldn't be to hard to understand.