mutual-auth: Support spire k8s service dns resolution

[sayboras]

Description

Please refer to individual commits for more details.

Testing

Testing with the temp commit, the conformance-e2e is running successfully.

https://github.com/cilium/cilium/actions/runs/5220420030/jobs/9423492566?pr=26031

[sayboras]

/test

[sayboras]

/test

[squeed]

If the operator needs to run on the same node as the SPIRE agent, then that should be expressed with PodAffinity. Or, we should include the spire container in the operator pod.

[squeed]

One point of confusion -- the PR seems to suggest that we need ClusterFirstWithHostNet if we wish to reach SPIRE via the service hostname. That also seems to be the default. So, do we need to set ClusterFirstWithHostNet automatically?

[sayboras]

If the operator needs to run on the same node as the SPIRE agent, then that should be expressed with PodAffinity. Or, we should include the spire container in the operator pod.

This was one of the approaches that we considered, however, cilium operator is running in HA mode with the leader election, so it's hard to control the node for both cilium operator and spire server.

[sayboras]

One point of confusion -- the PR seems to suggest that we need ClusterFirstWithHostNet if we wish to reach SPIRE via the service hostname. That also seems to be the default. So, do we need to set ClusterFirstWithHostNet automatically?

Sorry for the confusion, ideally, we would love to set ClusterFirstWithHostNet by default, however, such a configuration was not working in EKS due to #24774. Hence, the goal of this PR is to avoid setting dnsPolicy as ClusterFirstWithHostNet by default if mutual auth is enabled. Users can still have the option to set dnsPolicy in their deployment.

Let me update the commit message to make it clearer.

[zacharysarah]

LGTM for docs

[sayboras]

Force push to address the log comment

Diff

diff --git a/operator/auth/spire/client.go b/operator/auth/spire/client.go
index 6af073b4c6..6d59e299e3 100644
--- a/operator/auth/spire/client.go
+++ b/operator/auth/spire/client.go
@@ -173,18 +173,18 @@ func (c *Client) connect(ctx context.Context) (*grpc.ClientConn, error) {
 
 	tlsConfig := tlsconfig.MTLSClientConfig(source, source, tlsconfig.AuthorizeMemberOf(trustedDomain))
 
-	c.log.WithFields(map[string]interface{}{
-		logfields.URL:    c.cfg.SpireServerAddress,
-		logfields.IPAddr: resolvedTarget,
+	c.log.WithFields(logrus.Fields{
+		logfields.Address: c.cfg.SpireServerAddress,
+		logfields.IPAddr:  resolvedTarget,
 	}).Info("Trying to connect to SPIRE server")
 	conn, err := grpc.Dial(*resolvedTarget, grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig)))
 	if err != nil {
 		return nil, fmt.Errorf("failed to create connection to SPIRE server: %w", err)
 	}
 
-	c.log.WithFields(map[string]interface{}{
-		logfields.URL:    c.cfg.SpireServerAddress,
-		logfields.IPAddr: resolvedTarget,
+	c.log.WithFields(logrus.Fields{
+		logfields.Address: c.cfg.SpireServerAddress,
+		logfields.IPAddr:  resolvedTarget,
 	}).Info("Connected to SPIRE server")
 	return conn, nil
 }
diff --git a/pkg/logging/logfields/logfields.go b/pkg/logging/logfields/logfields.go
index c4796bf951..3dcd56051d 100644
--- a/pkg/logging/logfields/logfields.go
+++ b/pkg/logging/logfields/logfields.go
@@ -121,6 +121,9 @@ const (
 	// NextHop is an IPV4 or IPv6 address for the next hop
 	NextHop = "nextHop"
 
+	// Address is an IPV4, IPv6 or FQDN address
+	Address = "address"
+
 	// IPAddr is an IPV4 or IPv6 address
 	IPAddr = "ipAddr"
 

[sayboras]

/test

[sayboras]

/test

[sayboras]

Force pushed to rebase with main, and drop the last commit as cilium-cli version got upgraded in main branch now.

[sayboras]

/test

[sayboras]

Reviews from all required teams are in, all tests passed, merging.