k8s Central management

[elevran]

The goal is to consider how our object model can be mapped into k8s to provide a centralized management function that supports user management, security, persistency, etc.

In essence, we need to identify and define a relevant set of CRDs as well as model their creation and management using k8s facilities (e.g., namespace, RBAC).

A potential mapping would have

• a Fabric CRD (name, root CA, etc.) - cluster scoped object
• a Peer/Site CRD (name, certificates, gateway addresses) - these are cluster scoped and create a corresponding namespace. Parts can be read only so other peers can learn of remote's name, gateways etc. In such case, we could move the certificates as secrets to the namespace representing the peer
• Import/Export/Bindings are per namespace CRDs and are responsible for configuring the peer identified by the namespace (note that we could also explore the use of Kubernetes Multcluster Service (MCS) Export/Import objects in some fashion).

The mapping is relatively straightforward, but limits some of the more advanced checks and use cases we may want to explore. Most of these have to do with k8s RBAC based on verb+kind in a specific namespace, whereas we may want to control per specific attributes.

The could include, for example:

• restricting which peers you can import a service from (can be emulated by egress/ingress policies)
• limiting which Services can be exported
• scaling to multiple fabrics/sites (e.g., when managing multiple fabrics for namespace-scoped ClusterLink)
• etc.

The above can be built atop of KubeStellar as the distribution where ClusterLink implements per cluster agent/operator (react to CRDs by calling API) and management hub agent (applies management logic such as creating namespaces).
The per cluster operator and CRDs can be developed to allow declarative/kubectl like management irrespective of the central management.