Document required iptables rules when using Antrea Proxy

[antoninbas]

When using Antrea Proxy, one would expect kube-proxy / iptables performance not to impact Pod-to-Service traffic performance, since this traffic is handled in OVS. However, in practice performance deteriorates as the number of Services increase. This is because encapsulated traffic needlessly visits the kube-proxy iptables rules, incurring high CPU usage. The UDP tunnels are also added needlessly to the connection tracking table.

To avoid this issue, @tnqn @wenyingd and @edwardbadboy suggest installing the following iptables rules:

iptables -t raw -I PREROUTING -p udp --dport 6081 -m addrtype --dst-type LOCAL -j NOTRACK
iptables -t raw -I OUTPUT -p udp --dport 6081 -m addrtype --src-type LOCAL -j NOTRACK

These rules added to the raw table will ensure that all Geneve traffic bypasses connection tracking and kube-proxy iptables rules. The specific rules will differ based on the encapsulation mode being used.

At the very least, this should be documented. We should also investigate whether these rules can be installed by Antrea programmatically to simplify usage, or if there is any potential adverse effect.

Credits go to @tnqn @wenyingd and @edwardbadboy for troubleshooting this issue.

[antoninbas]

@tnqn @wenyingd it is unclear to me whether there is anything that can be done for noEncap mode, without risking to affect routing?

[antoninbas]

@tnqn @wenyingd is there a plan to push any change (documentation or other) for 0.10? or should we push this to the next milestone?

[wenyingd]

@tnqn @wenyingd is there a plan to push any change (documentation or other) for 0.10? or should we push this to the next milestone?

I think the fix should be included in next release.

[tnqn]

Agree this could be in next release, this should only matter when there are many iptables rules and many short-live connections. A proper change may need to limit the destination IPs to Node IPs of the cluster only, otherwise UDP connections to another endpoint that happens to use geneve/vxlan UDP port may be affected if conntrack is desired.

[tnqn]

@tnqn @wenyingd it is unclear to me whether there is anything that can be done for noEncap mode, without risking to affect routing?

I thought about this, but haven't come up with an approach that could bypass conntrack only for Pod-to-Pod traffic going throught host network. It's because PREROUTING chain is the only chance we can add "NOTRACK" for routed packets, but we cannot identify Pod-to-Pod packets from real Pod-to-Pod packets and the response of Pod-to-Service packets.