MCS: use cluster ID for ns lookup on exported service. #12064
Conversation
ClusterID uniquely identifies a cluster. It is used as a prefix to nslookup exported services. For example: <clusterid>.<svc>.<ns>.svc.clusterset.local Signed-off-by: sp98 <sapillai@redhat.com>
use ClusterID of the k8s cluster while doing nslookup of the exported mon or osd service IP Signed-off-by: sp98 <sapillai@redhat.com>
|
testing: kubectl get cephblockpools.ceph.rook.io mirrored-pool -n rook-ceph -o jsonpath='{.status.mirroringStatus.summary}' |
| @@ -2207,6 +2207,10 @@ type MultiClusterServiceSpec struct { | |||
| // like Globalnet Submariner. | |||
| // +optional | |||
| Enabled bool `json:"enabled,omitempty"` | |||
|
|
|||
| // ClusterID uniquely identifies a cluster. It is used as a prefix to nslookup exported | |||
| // services. For example: <clusterid>.<svc>.<ns>.svc.clusterset.local | |||
There was a problem hiding this comment.
How does the user know what to set the clusterID to? Would it be found in their kube context? And if the k8s dns lookup can work with it, isn't there a way rook can look up the cluster ID?
There was a problem hiding this comment.
Based on the discussion with Submariner team, the cluster ID can be from the kubeconfig but can also be configured by the user with something different. So we can't rely on kubeconfig every time.
The clusterID can be found in kubectl get submariners.submariner.io -n submariner-operator submariner -ojsonpath={.spec.clusterID}
There was a problem hiding this comment.
Ok, so the user must be aware of the clusterID and apply it the same both for submariner and in rook.
| @@ -75,7 +75,7 @@ func (c *Cluster) createService(mon *monConfig) (*v1.Service, error) { | |||
|
|
|||
| func (c *Cluster) exportService(service *v1.Service, monDaemon string) (string, error) { | |||
| logger.Infof("exporting service %q", service.Name) | |||
| exportedIP, err := k8sutil.ExportService(c.ClusterInfo.Context, c.context, service) | |||
| exportedIP, err := k8sutil.ExportService(c.ClusterInfo.Context, c.context, service, c.spec.Network.MultiClusterService.ClusterID) | |||
There was a problem hiding this comment.
Where this is added for upstream?
There was a problem hiding this comment.
didn't get that question.
| @@ -2207,6 +2207,10 @@ type MultiClusterServiceSpec struct { | |||
| // like Globalnet Submariner. | |||
| // +optional | |||
| Enabled bool `json:"enabled,omitempty"` | |||
|
|
|||
| // ClusterID uniquely identifies a cluster. It is used as a prefix to nslookup exported | |||
| // services. For example: <clusterid>.<svc>.<ns>.svc.clusterset.local | |||
There was a problem hiding this comment.
Ok, so the user must be aware of the clusterID and apply it the same both for submariner and in rook.
MCS: use cluster ID for ns lookup on exported service. (backport #12064)
Description of your changes:
In MCS, to access the Service in a specific cluster, prefix the query with cluster-id so that we only query for the exported service on the local cluster.
clusterid is a unique identity/name for a cluster and it should not overlap with any of the remote clusters in the clusterset as per the MCS API.
ClusterIDshould be provided by the user because there are multiple ways in which the clusterid can be configured as part of Submariner installation and assuming one could break the solution when deployed with a custom clusteridWhich issue is resolved by this Pull Request:
Resolves #12065
Checklist:
skip-cion the PR.