Topology Aware Routing

[robscott]

Enhancement Description

• One-line enhancement description (can be used as a release note): EndpointSlice controller can be configured to provide topology aware hints that will enable simple topology aware routing.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2433-topology-aware-hints
• Discussion Link: https://docs.google.com/document/d/1ZzUoFY1SrdjVefl7gVOJZJLt1I1LHttw8pcX95nlgMY/edit
• Primary contact (assignee): @gauravkghildiyal
• Responsible SIGs: sig-network
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): v1.21
  • Beta release target (x.y): v1.23
  • Stable release target (x.y): v1.33
• Alpha
  • KEP (k/enhancements) update PR(s): KEP 2004: Evolving Topology Aware Routing for Services #2005
  • Code (k/k) update PR(s):
  • Docs (k/website) update PR(s):
• Beta
  • KEP (k/enhancements) update PR(s): KEP-2433: Removing traffic policy interoperability as beta graduation requirement #3043
  • Code (k/k) update PR(s):
    • 1.23 default false Graduate TopologyAwareHints feature gate to beta kubernetes#106433
    • 1.24 default true Updating TopologyAwareHints feature gate to be on by default kubernetes#108747
  • Docs (k/website) update PR(s): Mark that TopologyAwareHints feature is default on in 1.24 website#32361
• GA
  • KEP (k/enhancements) update PR(s):
    • 1.26 Updating Topology Aware Hints KEP with GA Graduation Criteria #3572
    • 1.27 KEP-2413: Graduate SeccompDefault to stable #3718, KEP-2433 Topology Aware Hints: Adding SameZone heuristic and other tweaks #3765
    • 1.28 KEP-2433: add new heuristic to topology routing #4003
    • 1.29-1.30 KEP-2433: Bumping stable target to 1.30 #4284
  • Code (k/k) update PR(s):
    • 1.29 TopologyAwareHints: Take lock in HasPopulatedHints kubernetes#118189
  • Docs (k/website) update PR(s):

/sig network

[robscott]

/assign

[kendallroden]

Hi @robscott ,
Since your Enhancement is scheduled to be in 1.21, please keep in mind the important upcoming dates:

• Tuesday, March 9th: Week 9 - Code Freeze
• Tuesday, March 16th: Week 10 - Docs Placeholder PR deadline
  • If this enhancement requires new docs or modification to existing docs, please follow the steps in the Open a placeholder PR doc to open a PR against k/website repo.
• As a reminder, please link all of your k/k PR(s) and k/website PR(s) to this issue so we can track them.

Thanks!

[JornShen]

@robscott What' the develop process of this feature? I am interested in it. Is there something I can help to do ? :)

[robscott]

Hey @JornShen thanks for checking in! If you have time, I'd really appreciate any review or testing of my related PR: kubernetes/kubernetes#99522.

[kendallroden]

Hey @robscott! I know you are aware based on other enhancement work that code freeze is coming up on March 9th EOD PST and if any PRs are not merged by the deadline, you'll have to request an exception! Please also keep in mind that if this enhancement requires new docs or modification to existing docs, you'll need to follow the steps in the Open a placeholder PR doc to open a PR against k/website repo by March 16th EOD PST
Thanks!

[annajung]

Hi @robscott, with kubernetes/kubernetes#99522 merged in, we will mark this as code complete for 1.21 release.

[reylejano]

Hello @robscott , 1.21 Docs lead here.
Does this enhancement work planned for 1.21 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.21 branch in the k/website repo. This PR can be just a placeholder at this time and must be created by March 16 EOD PST
Also take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[robscott]

Hey @reylejano, thanks for checking in! This will require some significant docs additions. I'll make sure to have a docs PR ready before that deadline.

[JamesLaverack]

/milestone v1.22
/stage beta

[JamesLaverack]

Hey @robscott, 1.22 Enhancements Lead here,

I have a few questions about your enhancement, as it is currently targeted for 1.22.

• Your kep.yaml needs updating to reflect the graduation. The latest-milestone should be set to "v1.22".

I'm assuming you're targeting beta for this release (if you're not please let me know!) in which case we will also require:

• Your KEP must list graduation critera for beta.
• You will require a beta PRR review from the PRR team.

We require these merged into master by enhancements freeze at 23:59:59 on Thursday 13th May, or else require an exception after this deadline.

Please do reach out if you have any questions.

[robscott]

Hey @JamesLaverack, sorry I missed this! The KEP updates for 1.22 have merged. The PRR was actually already completely for the alpha release.

Relevant KEP PRs that have merged:

• Updating Topology Hints KEP for 1.22 #2706
• Updating topology stage to beta for PRR approval #2714

[jonathon2nd]

Checked in to see if there has been any update to allow for previous behavior.

Based of what I see here and the various other PR's and issues, it looks like a no?

Adding image as https://kubernetes.io/docs/concepts/services-networking/service-topology/ has been deleted.

[aojea]

Checked in to see if there has been any update to allow for previous behavior.

see https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/4444-service-routing-preference

[thockin]

@robscott what's the GA pln for this?

[robscott]

@robscott what's the GA pln for this?

The outcome here is closely tied to #4444, as it's a natural evolution of part of this KEP.

This KEP (#2433) essentially bundled two things together - the EndpointSlice hints field and a Service-level annotation that could be used to automatically populate hints with a rather confusing heuristic.

#4444 continues to use the hints field, but adds a new trafficDistribution field and a simpler heuristic instead of the annotation and confusing heuristic. Maybe this table is a better summary:

	#2433	#4444
Hints	Defined in this KEP	Relied on by this KEP
Opt-In	service.kubernetes.io/topology-mode Annotation	trafficDistribution Field
Heuristic	Complicated	Simple

My recommendation would be to keep the hints field defined in #2433 and the trafficDistribution field and simpler heuristic from #4444.

If we're good with that plan, I think that when #4444 graduates to beta (likely v1.31) we should formally deprecate the annotation and corresponding heuristic defined by this KEP. Then we can work to graduate #2433 to GA as a hints-only KEP and #4444 to GA as a KEP covering the trafficDistribution field and simple heuristic.

/cc @gauravkghildiyal

[danwinship]

/label lead-opted-in
/milestone v1.33

targeting GA for just the hints field in 1.33, as per Rob's comment

[ArkaSaha30]

Hello @robscott 👋, v1.33 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th February 2025 / 19:00 PDT Thursday 13th February 2025.

This enhancement is targeting stage stable for v1.33 (correct me, if otherwise)
/stage stable

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.33. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 6th February 2025 so that the PRR team has enough time to review your KEP.

For this KEP, we would just need to update the following:

• KEP needs to be updated with the target stable for latest-milestone: v1.33
• PRR needs to be updated with the stable approvers and needs to be mentioned in KEP as prr-approvers:

The status of this enhancement is marked as At risk for enhancements freeze. Please keep the issue description up-to-date with appropriate stages as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[gauravkghildiyal]

Thanks for the reminder @ArkaSaha30. I'll be helping out with the required changes and will update as we make progress.

[gauravkghildiyal]

/cc

[robscott]

Thanks to @gauravkghildiyal for taking this one over!

/unassign
/assign @gauravkghildiyal

[dipesh-rawat]

Hi @gauravkghildiyal 👋, 1.33 Enhancements team here,

Just a quick friendly reminder as we approach the enhancements freeze later this week, at 02:00 UTC Friday 14th February 2025 / 19:00 PDT Thursday 13th February 2025.

The current status of this enhancement is marked as At risk for enhancement freeze. There are a few requirements mentioned in the comment #2433 (comment) that still need to be completed.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[dipesh-rawat]

Hello @gauravkghildiyal 👋, 1.33 Enhancements team here,

Now that PR #5151 has been merged, all the KEP requirements are in place and merged into k/enhancements.

Before the enhancement freeze, it would be appreciated if following nit could be addressed:

• Update KEP (k/enhancements) update PR(s) section in the issue description to include the PR KEP-2433: Graduate Topology Aware Hints with only the Hints field to GA #5151

Aside from the minor nit mentioned above, this enhancement is all good for the upcoming enhancements freeze. 🚀

The status of this enhancement is now marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!
(cc: @ArkaSaha30)

/label tracked/yes

[rayandas]

Hello @gauravkghildiyal 👋, v1.33 Docs Lead here.

Does this enhancement work planned for v1.33 require any new docs or modification to existing docs?

If so, please follow the steps here to open a PR against dev-1.33 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 27th February 2025 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[gauravkghildiyal]

Hi @rayandas -- Opened kubernetes/website#49914

[rayandas]

Thanks @gauravkghildiyal
Please update the description as well.

[rytswd]

Hi @gauravkghildiyal 👋 -- this is Ryota (@rytswd) from the 1.33 Communications Team!

For the 1.33 release, we are currently in the process of collecting and curating a list of potential feature blogs, and we'd love for you to consider writing one for your enhancement!

As you may be aware, feature blogs are a great way to communicate to users about features which fall into (but not limited to) the following categories:

• This introduces some breaking change(s)
• This has significant impacts and/or implications to users
• ...Or this is a long-awaited feature, which would go a long way to cover the journey more in detail 🎉

To opt in to write a feature blog, could you please let us know and open a "Feature Blog placeholder PR" (which can be only a skeleton at first) against the website repository by Wednesday, 5th March, 2025? For more information about writing a blog, please find the blog contribution guidelines 📚

Tip

Some timeline to keep in mind:

• 02:00 UTC Wednesday, 5th March, 2025: Feature blog PR freeze
• Monday, 7th April, 2025: Feature blogs ready for review
• You can find more in the release document

Note

In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[rytswd]

Hi @gauravkghildiyal 👋, 1.33 Communications Team here again!

This is a gentle reminder for the feature blog deadline mentioned above, which is 02:00 UTC Wednesday, 5th March, 2025. To opt in, please let us know and open a Feature Blog placeholder PR against k/website by the deadline. If you have any questions, please feel free to reach out to us!

Tip

Some timeline to keep in mind:

• 02:00 UTC Wednesday, 5th March, 2025: Feature blog PR freeze
• Monday, 7th April, 2025: Feature blogs ready for review
• You can find more in the release document

Note

In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[ArkaSaha30]

Hello @robscott @gauravkghildiyal 👋, v1.33 Enhancements team here.

With all the implementation (code-related) PRs merged per the issue description:

• TopologyAwareHints: Take lock in HasPopulatedHints kubernetes#118189

This enhancement is now marked as Tracked for code freeze for the {current release} Code Freeze!

Additionally, please let me know if there are any other PRs in k/k that we should track for this KEP, so that we can maintain accurate status.

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.