GAMMA: multi-cluster mesh

[shaneutt]

In Kuma we have the concept of making a single mesh across multiple clusters. I am curious if the multi-cluster mesh use case feels important to other implementations, and if so would that be valuable across different mesh implementations, e.g. for GAMMA something like a Mesh resource which you can decorate to connect to another Mesh resource that lives in another cluster, so as to "join" it? Just generally curious for thoughts.

[costinm]

Istio is doing this automatically - K8S is using ServiceExport.

IMO GAMMA should probably support at least those 2 modes - not sure if we want more granularity or alternatives.

[brianehlert]

To me this almost sounds like a 'federation' - how do two meshes participate with each other. Whether they share a higher level management plane or not.
The Istio assumption might be that each cluster is an individual but joined through higher level abstractions.
The Kuma assumption might be that they can achieve something similar without the management plane overhead
(I don't know the implementations in detail, I am making examples)
But there is no reason why an NGINX Mesh couldn't talk to a Kuma Mesh if you really wanted to take it all the way.

[keithmattix]

FWIW, I think this is actually distinct from federation. If two clusters "join" (that name isn't great) the same Mesh resource, they are part of the same mesh for the purposes of routing. I'm not sold that we want to actually do this in GAMMA though

[mikemorris]

As a recent update, GEP-1748 is proposing the adoption of MCS API for routing to services on other clusters within a clusterset (a group of clusters which has an assumption of sameness) by specifying a ServiceImport resource as a backendRef to route to Services exposed from any cluster in the clusterset by a ServiceExport.

This doesn't yet address how to route to off-cluster services outside the local clusterset, but I'm hoping this is a good starting point and we can eventually work to expand the MCS API to support "cross-clusterset" routing in the future.

[brianehlert]

I still see multiple methods for routing.
Take for example RedHat's recent announcement around skupper.io
It is not a mesh as we generally talk about it, but it addressing the cross-cluster routing problem in its own way. And leaving a mesh to worry about in-cluster concerns.
Makes me always wonder how far in carrying forward assumptions of current products and current implementations vs. leaving opportunity for new solutions.