mesh-gateway podsecuritypolicy prevents using hostNetwork

[alexdulin]

While the mesh-gateway can be configured to use hostNetwork, and it gets correctly set in the deployment, it is always set to false in the podsecuritypolicy regardless of the configured values. This makes it impossible to use hostNetwork: true for the mesh gateways without setting enablePodSecurityPolicies: false and using out of band processes to create the PSPs.

It would be great if the podsecuritypolicy for mesh-gateways could not be hard coded to use hostNetwork: false. Without changing the PSP, what is the point of being able to set meshGateway.hostNetwork at all?

[david-yu]

@alexdulin Could you tell us a little bit more about why you are looking to host network? Are you using Cilium perhaps as your CNI?

[alexdulin]

We recently added a Consul datacenter on EKS as a secondary to a primary datacenter running on regular EC2 instances, and ever since we have been seeing error logs like the following on the EKS agents:

2020-09-15T22:54:00.791Z [ERROR] agent.server.memberlist.wan: memberlist: Failed to send indirect ping: write tcp [REDACTED]:54666->[REDACTED]:32010: write: broken pipe from=[REDACTED]:8302
2020-09-15T22:54:11.331Z [ERROR] agent.server.memberlist.wan: memberlist: Failed to send ack: write tcp [REDACTED]:35446->[REDACTED]:32010: write: broken pipe from=[REDACTED]:41570
2020-09-15T22:54:18.692Z [ERROR] agent.server.memberlist.wan: memberlist: Failed to send ping: write tcp [REDACTED]:59088->[REDACTED]:32010: write: broken pipe
2020-09-15T22:54:21.123Z [ERROR] agent.server.memberlist.wan: memberlist: Failed to send ack: write tcp [REDACTED]:34938->[REDACTED]:32010: write: broken pipe from=[REDACTED]:37824

Everything seems to be working correctly - ACL replication and cross-datacenter communication via the mesh gateways are both working - but these error messages are worrying. After a lot of digging to try and trace the source of the problem I wanted to test if it was due to some issue with kube-proxy not fully sending along TCP half-close. Using the hostNetwork seems like a good first step to removing kube-proxy from the possible list of culprits.

If you have other suggestions for where those errors are coming from that would be great, but it still does not solve the fact that the templates do not support using hostNetwork for the mesh gateways with the PSP in its current state.

Edit: We are using mesh gateways for WAN federation.