Enable per-endpoint routes by default

[pchaigno]

Per-endpoint routes (enable-endpoint-routes) are currently disabled by default. We want to enable them by default for 1.10, mostly as a workaround for #10645. This is a meta issue to track progress on that work item.

• Fix VXLAN + per-endpoint routes bug, Connectivity loss through LB with vxlan + per-endpoint routes #14646 - bpf: Do not bypass conntrack if running kube-proxy+vxlan+endpoint routes #14675
• Fix VXLAN + per-endpoint routes bug v2, Packets to pods processed twice with per-endpoint routes + VXLAN #14657.
• Fix SNAT bypass in IPv6 - iptables: Fix incorrect SNAT bypass with endpoint routes and tunneling #14913
• Fix fragment tracking, Fragment tracking doesn't work with endpoint routes #15958.
• Fix reverse DNAT with tunneling and kube-proxy, VXLAN + kube-proxy + endpoint routes breaks policy enforcement for services #16446.
• Support host firewall with per-endpoint routes, Enforce host policies when using per-endpoint routes #13121 - Support host policies with per-endpoint routes #15217
• Support BPF host routing with per-endpoint routes, cilium: endpoint routes + veth fast redirect support #14240.
• Fix ICMPv6 NS drops, datapath: Per endpoint routes are broken IPv6 connectivity between pods on the same node when netpol is applied #23852.
• CI run with per-endpoint routes enabled - defaults: Enable per-endpoint routes by default #15057
• Evaluate performance impact of per-endpoint routes with large number of endpoints on node.
• Enable by default.
• Clean up/revert CI code that handles endpoint-routes reconfiguration - See comment in commit in test: Redeploy DNS after endpointRoutes reconfiguration #16767.

[joestringer]

Just linking in issue #16717 here, seemed like a case where if we had a mix of endpoints with endpoint-routes enabled + endpoint-routes disabled, it could cause forwarding issues between the endpoints. One longer-term approach to solving that issue is to support endpoints in different modes concurrently. An alternative approach would be to support endpoint routes with all other configurations (per the goal of this issue) and switch to this by default.

[m-yosefpor]

are there any plans to have this in 1.13 milestone?

[pchaigno]

Hi @m-yosefpor 👋 You should already be able to enable endpoint routes in your cluster if you need it. Or is there a reason you need it to be enabled by default?

[m-yosefpor]

Hi @pchaigno . Actually the problem is endpoint routes cannot be enabled together with BPF host routing. Having endpoint routes enabled by default is not our requirement, but one of the items listed here in the check list (#14240) is essential for us.

[pchaigno]

Ok 👍 As far as I know, there are no plans to address #14240 for v1.12 or v1.13. It's something we'd like to address but it's not prioritized right now.

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

[gandro]

Another discrepancy: #28812