Trace works about enable the ipvs mode for the kube-proxy by default

[Lion-Wei]

This issue is to trace the works about make IPVS mode as kube-proxy default mode.
/kind feature

• kube-proxy IPVS spurious logs removing IPv6 address. kube-proxy IPVS spurious logs removing IPv6 address #70113
• kube-proxy ipvs auto detect network interface mtu. kube-proxy ipvs auto detect network interface mtu #68546 (WIP)
• In ipv6-only cluster with proxy-mode=ipvs ipv4 entries are used for NodePort. In ipv6-only cluster with proxy-mode=ipvs ipv4 entries are used for NodePort #68437 (WIP)
• Unable use localhost: with IPVS. Unable use localhost:<nodeport> with IPVS #67730 (WIP)
• IPVS is not working with hostport. IPVS is not working with hostport  #66103
• Make IPVS as default kube-proxy mode.

@luxas @timothysc @m1093782566 @bowei @caseydavenport @cmluciano , Please check.

[Lion-Wei]

/area ipvs
/sig network
/kind feature
/assign

[m1093782566]

cc @thockin @rramkumar1

[Lion-Wei]

#68974 and #65820 are solved issue, so I didn't put them on the list, seems both those two issue can be closed. : )

[timothysc]

Are you intending to target 1.14?

[uablrek]

#71596 is related

[luxas]

Are you intending to target 1.14?

ping @Lion-Wei

[m1093782566]

Yes, we are!

[luxas]

Ok. We need to extensively build out e2e testing for this then.

[luxas]

cc @kubernetes/sig-testing @kubernetes/sig-cluster-lifecycle @kubernetes/sig-network-feature-requests @kubernetes/sig-architecture-feature-requests for visibility, as this is significant

[luxas]

IMO we need to create a new tracking issue in https://github.com/kubernetes/enhancements for this, and (maybe) a KEP how this transition will happen smoothly across all clusters, what the tradeoffs/migration steps are, etc.

[uablrek]

If ipv6 is considered #65006 is related also, and hard to fix since the problem is in a vendor library.

[BenTheElder]

not sure what coverage we have for this currently (had to track it all!), but we definitely need thorough e2e coverage if we're going to flip defaults. FYI @spiffxp

[m1093782566]

There are already some test grids for IPVS proxier, they are running all network-sig e2e tests.

[dghubble]

Some of the issues listed (in particular, not working with hostport #66103) have hindered adopting IPVS in clusters at all. Since these weren't included in GA, is there some other maturation window envisioned?

[justaugustus]

Given @dghubble's feedback, I'd like to see a KEP for this that includes well-defined graduation criteria.

[luxas]

Yes, let's make sure we have an absolute 100% feature parity before we switch. Working with ipv6 going forward is also a necessity. A switch of a default is definitely KEP-worthy IMO, after seeing these comments.

[uablrek]

In the ipv4/ipv6 dual-stack kep kubernetes/enhancements#648 ipvs support is still only "nice-to-have". If proxier ipvs is default one may have to "downgrade" to iptables to get dual stack.
IMO ipvs should be supported also for dual stack.

[Quentin-M]

#71071

[luxas]

@uablrek Yes, but in ipv6-only cluster ipvs should still work out-of-the-box without a "downgrade" to iptables, right?

Also I noticed this fun thing that ipvs is marked as experimental still in the CLI:

--proxy-mode ProxyMode
    Which proxy mode to use: 'userspace' (older) or 'iptables' (faster) or 'ipvs' (experimental). If blank, use the best-available proxy (currently iptables).  If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.

[nikopen]

Greetings!

🏔️ code freeze 🏔️ is coming in about 10 days, is this intented to be implemented in the following 2-3 weeks?

@m1093782566 @Lion-Wei

[soggiest]

Hello! Code freeze for 1.14 is coming up in 2 days. Will this be implemented in the next week? Or should we punt until 1.15?

@m1093782566 @Lion-Wei

[m1093782566]

Sound fair, we can put it to 1.15 cycle.

[soggiest]

/milestone 1.15

[k8s-ci-robot]

@soggiest: The provided milestone is not valid for this repository. Milestones in this repository: [next-candidate, v1.10, v1.11, v1.12, v1.13, v1.14, v1.15, v1.16, v1.17, v1.18, v1.4, v1.5, v1.6, v1.7, v1.8, v1.9]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[soggiest]

/milestone clear

[uablrek]

It must be clearified that proxy-mode=ipvs does not work in an ipv6-only cluster.

The support for ipv6 in proxy-mode=ipvs is re-classified as a "feature" in #65006

[uablrek]

@luxas

About #71202 (comment), apparently proxy-mode=ipvs was not intended to work with ipv6.

[dims]

/remove-sig architecture

please re-tag arch if needed, looks like all items fall under sig-network

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[neolit123]

/lifecycle frozen

[rikatz]

/area kube-proxy

[rikatz]

So we've been discussing about the evolution of kube-proxy and the maintainability of the modes (IPtables, ipvs, and so on) and we've reached a consensus that we need to evolve the vendoring of kube-proxy logics, modules (also take a look at: https://github.com/kubernetes-sigs/kpng)

So right now, talking with other sig-network folks, (@thockin @aojea @jayunit100 @uablrek ) we think this is not a movement we want to do right now (defaulting to IPVS) so I'm closing this issue :)

Thanks!

/close

[k8s-ci-robot]

@rikatz: Closing this issue.

In response to this:

So we've been discussing about the evolution of kube-proxy and the maintainability of the modes (IPtables, ipvs, and so on) and we've reached a consensus that we need to evolve the vendoring of kube-proxy logics, modules (also take a look at: https://github.com/kubernetes-sigs/kpng)

So right now, talking with other sig-network folks, (@thockin @aojea @jayunit100 @uablrek ) we think this is not a movement we want to do right now (defaulting to IPVS) so I'm closing this issue :)

Thanks!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.