Topology Aware Routing

[robscott]

Enhancement Description

• One-line enhancement description (can be used as a release note): EndpointSlice controller can be configured to provide topology aware hints that will enable simple topology aware routing.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2433-topology-aware-hints
• Discussion Link: https://docs.google.com/document/d/1ZzUoFY1SrdjVefl7gVOJZJLt1I1LHttw8pcX95nlgMY/edit
• Primary contact (assignee): @robscott
• Responsible SIGs: sig-network
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): v1.21
  • Beta release target (x.y): v1.23
  • Stable release target (x.y): v1.28
• Alpha
  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update PR(s):

/sig network

[robscott]

/assign

[kendallroden]

Hi @robscott ,
Since your Enhancement is scheduled to be in 1.21, please keep in mind the important upcoming dates:

• Tuesday, March 9th: Week 9 - Code Freeze
• Tuesday, March 16th: Week 10 - Docs Placeholder PR deadline
  • If this enhancement requires new docs or modification to existing docs, please follow the steps in the Open a placeholder PR doc to open a PR against k/website repo.
• As a reminder, please link all of your k/k PR(s) and k/website PR(s) to this issue so we can track them.

Thanks!

[JornShen]

@robscott What' the develop process of this feature? I am interested in it. Is there something I can help to do ? :)

[robscott]

Hey @JornShen thanks for checking in! If you have time, I'd really appreciate any review or testing of my related PR: kubernetes/kubernetes#99522.

[kendallroden]

Hey @robscott! I know you are aware based on other enhancement work that code freeze is coming up on March 9th EOD PST and if any PRs are not merged by the deadline, you'll have to request an exception! Please also keep in mind that if this enhancement requires new docs or modification to existing docs, you'll need to follow the steps in the Open a placeholder PR doc to open a PR against k/website repo by March 16th EOD PST
Thanks!

[annajung]

Hi @robscott, with kubernetes/kubernetes#99522 merged in, we will mark this as code complete for 1.21 release.

[reylejano]

Hello @robscott , 1.21 Docs lead here.
Does this enhancement work planned for 1.21 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.21 branch in the k/website repo. This PR can be just a placeholder at this time and must be created by March 16 EOD PST
Also take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[robscott]

Hey @reylejano, thanks for checking in! This will require some significant docs additions. I'll make sure to have a docs PR ready before that deadline.

[JamesLaverack]

/milestone v1.22
/stage beta

[JamesLaverack]

Hey @robscott, 1.22 Enhancements Lead here,

I have a few questions about your enhancement, as it is currently targeted for 1.22.

• Your kep.yaml needs updating to reflect the graduation. The latest-milestone should be set to "v1.22".

I'm assuming you're targeting beta for this release (if you're not please let me know!) in which case we will also require:

• Your KEP must list graduation critera for beta.
• You will require a beta PRR review from the PRR team.

We require these merged into master by enhancements freeze at 23:59:59 on Thursday 13th May, or else require an exception after this deadline.

Please do reach out if you have any questions.

[ialidzhikov]

Shouldn't we tackle first kubernetes/kubernetes#113731 before promoting to GA? We are still looking for the simplest strategy/heuristic named SameZone - if there are endpoints in the same zone, route to those Endpoints, if not - fallback to any available Endpoint. I believe we are not the only ones out there in the community requesting this feature.

[thockin]

I don't think this GA needs to block on the alternative heuristics. The parts that are covered by the KEP now are "done", I think, and we can either open a new KEP to cover additional heuristics (and then argue over the need to port them to different implementations) or just proceed to PRs. The real problem here is that nobody has stepped up to implement the alternative heuristic(s) yet, and in the meantime this KEP *is* valuable to some people.

…

On Fri, May 12, 2023 at 6:06 AM Ismail Alidzhikov ***@***.***> wrote: Shouldn't we tackle first kubernetes/kubernetes#113731 <kubernetes/kubernetes#113731> before promoting to GA? We are still looking for the simplest strategy/heuristic named SameZone - if there are endpoints in the same zone, route to those Endpoints, if not - fallback to any available Endpoint. I believe we are not the only ones out there in the community requesting this feature. — Reply to this email directly, view it on GitHub <#2433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKWAVFRHJP542FQXNTAVU3XFYYUFANCNFSM4XEDSTJQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[aojea]

@thockin @ialidzhikov @robscott added a new heuristic that simplify existing one but still has some safeguards to avoid blackholing traffic #4003

[thockin]

Do we want to block this GA on the 2nd heuristic, or do we GA this and start a new one?

[thockin]

I need a decision on this for 1.28

[robscott]

I think @aojea was working on an update to the KEP PR that would simplify the changes proposed for v1.28. I personally would prefer to wait for GA until we have one or more of the following:

1. An alternative heuristic that is simpler and more predictable (I believe that's what @aojea is updating the proposal to cover)
2. A clearer path to bringing your own hints approach (likely related to Move EndpointSlice Reconciler into Staging #3685)

[thockin]

Moving to 1.29

[AnaMMedina21]

Hola @robscott 👋, v1.29 Enhancements team here.

I am just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2023..

This enhancement is targeting for stage stable for v1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: { v1.29}. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, we would just need to update the following:

• For the PRR to be updated with approval for stable

The status of this enhancement is marked as At Risk for Enhancements Freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[jeremyrickard]

The PRR could use another pass for graduating to stable, and is missing a new section in the stability section:

Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?

I had a question around the SLO while reading the current version of the PRR template.

What are the reasonable SLOs (Service Level Objectives) for the above SLIs? As a starting point, it is likely reasonable for the EndpointSlice controller to experience up to a 10% sync failure rate. This is largely related to it trying to update stale EndpointSlices. When we are able to find a solution for that issue the expected sync failure rate should be significantly lower. This specific problem is most notable for large Services that have rapidly updating endpoints.

Have you been able to find a solution for the issue around updating stale EndpointSlices? A 10% sync failure rate seems somewhat high for a stable feature. Is that still a good target for understanding if this should be disabled?

[AnaMMedina21]

Hey @robscott 👋, v1.29 Enhancements team here, checking in once more as we approach the v1.29 enhancement freeze deadline on 01:00 UTC, Friday, 6th October 2023. The status of this enhancement is marked as At Risk for Enhancements Freeze. It looks like the PRR needs another pass before approval. Let me know if I missed anything. Thank you!

[robscott]

Hey @AnaMMedina21 and @jeremyrickard, sorry for losing track of this! To clarify, we're not actually trying to make any changes to the KEP this cycle. We ran out of time to complete the changes we wanted to in 1.28 so those have just carried over to 1.29. We're continuing to stay in "beta", which I think means our PRR and everything else should still be good to go:

enhancements/keps/prod-readiness/sig-network/2433.yaml

Lines 1 to 5 in 0d7e982

	kep-number: 2433
	alpha:
	approver: "@wojtek-t"
	beta:
	approver: "@wojtek-t"

Definitely let me know if I'm missing anything though.

[npolshakova]

Hello 👋, 1.29 Enhancements Lead here.
Unfortunately, this enhancement did not meet requirements for v1.29 enhancements freeze.
Feel free to file an exception to add this back to the release tracking process. Thanks!

/milestone clear

[robscott]

@npolshakova can you clarify what we were missing for this milestone? We're not making any changes to the KEP in this cycle, just trying to get the implementation caught up to the KEP... I asked for clarification on this 2 days ago in the comment above and didn't get a response.

[npolshakova]

Hi @robscott, if you have updated the alpha criteria then you do need this enhancement tracked for 1.29. If you intend to just address bug fixes in this release, this does not need to be tracked by the enhancements team for 1.29.

For the enhancement team to track this KEP we need a couple minor updates:

• latest-milestone should be updated to 1.29
• The stable milestone should be removed or bumped from 1.29

Let me know if you have any further questions. Thanks!

[robscott]

Thanks for catching that @npolshakova! I've filed #4284 to update the KEP metadata here. I don't think I can remove the stable milestone from this issue, but we did not intend to target stable in this release, sorry for the confusion.

[robscott]

@npolshakova are you able to update the labels on this issue now that #4284 has merged? Anything else I'm missing?