Allow informers for getting a stream of data instead of chunking

[p0lyn0mial]

Enhancement Description

• One-line enhancement description (can be used as a release note): informers can get a stream of data to prime the caches
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/3157-watch-list/README.md
• Discussion Link:
• Primary contact (assignee): @p0lyn0mial
• Responsible SIGs: sig-apimachinery, sig-scalability
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.27
  • Beta release target (x.y): 1.32
  • Stable release target (x.y): TBD
• Alpha
  • KEP (k/enhancements) update PR(s):
    • KEP-3157: allow informers for getting a stream of data instead of chunking #3142
    • Update KEP 3157 for milestone 1.27 #3822
  • Code (k/k) update PR(s):
    • WIP: POC introduce API streaming kubernetes#106477
    • e2e/apimachinery: Add an e2e test for the API Streaming feature kubernetes#117277
    • cacher avoid double locking kubernetes#117410
  • Docs (k/website) update PR(s):
    • Streaming lists documentation website#39993
• Beta
  • KEP (k/enhancements) update PR(s):
    • 1.30 KEP 3157 (watch-list): add a new beta criterion #4453
    • 1.32 Update KEP 3157 (watch-list) for milestone 1.32 #4836
  • Code (k/k) update PR(s):
    • 1.30 storage/cacher: ensure the cache is at the Most Recent ResourceVersion when streaming was requested kubernetes#122830
    • 1.31 cleanup: replace ENABLE_CLIENT_GO_WATCH_LIST_ALPHA with WatchListClient gate kubernetes#122791
    • 1.32:
      • client-go/rest/request: decodes initialEventsListBlueprint for watchlist requests kubernetes#127812
      • apiserver/handlers/watch: encode initialEventsListBlueprint with watchEncoder kubernetes#127587
      • cacher: prevents sending events with ResourceVersion < RequiredResourceVersion kubernetes#127500
      • metadata client: add support for API streaming  kubernetes#127388
      • apimachinery/meta/types.go: define InitialEventsListBlueprintAnnotationKey const kubernetes#127319
      • cacher: apply key for initial events only if the call is not recursive kubernetes#127265
      • adds watchListEndpointRestrictions for watchlist requests kubernetes#126996
      • Promote WatchList feature to Beta  kubernetes#128053
  • Docs (k/website) update(s):
    • WIP: Document KEP 3157 (watch-list) for milestone 1.30 website#45283
    • WIP: Add a blog post about using the API streaming (KEP 3157) website#45285
    • WIP: Add a blog post introducing feature gates for client-go website#45286
    • Add feature gate WatchListClient for v1.30 website#47261
    • Document KEP 3157 (watch-list) for milestone 1.32 website#48512
    • Blog post: Enhancing Kubernetes API Server Efficiency with API Streaming website#48513

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

/sig api-machinery
/sig scalability

[gracenng]

Hi @p0lyn0mial ! 1.24 Enhancements team here. Just checking in as we approach enhancements freeze on 18:00pm PT on Thursday Feb 3rd. This enhancements is targeting alpha for 1.24
Here’s where this enhancement currently stands:

• Updated KEP file using the latest template has been merged into the k/enhancements repo. KEP-3157: allow informers for getting a stream of data instead of chunking #3142
• KEP status is marked as implementable for this release with latest-milestone: 1.24
• KEP has a test plan section filled out.
  - [ ] KEP has up to date graduation criteria. -- not needed for alpha
• KEP has a production readiness review that has been completed and merged into k/enhancements -- PRR not found

The status of this enhancement is track as at risk
Thanks!

[p0lyn0mial]

Hi @gracenng! Many thanks for the heads-up. Could you please double-check the last point? The KEP has the Production Readiness Review Questionnaire. In addition to that, I have filled the required bits for the alpha version - it just hasn't been reviewed yet.

[gracenng]

Hi @p0lyn0mial,
Thanks for the reply. Your PRR Questionaire in the KEP needs to be approved by the PRR team, which will result in a file in
https://github.com/kubernetes/enhancements/tree/master/keps/prod-readiness/
Your sig has not made a folder there yet, please check out the format.

Regarding your second point, I am tracking your KEP files and the first bullet will be checked when the PR is merged. Does that answer your question?

[gracenng]

Hi @p0lyn0mial , 1.24 Enhancements Team here.

Reaching out as we're less than a week away from Enhancement Freeze on Thursday, February 3rd.
There's no update for this enhancement since last checkin, let me know if I missed anything.
Current status is at risk

[p0lyn0mial]

Hi @gracenng, it looks like it won't make it to 1.24. Please leave me some instructions on how to update the KEP so it won't show up on your query.

[gracenng]

No worries I’ll take care of it

…

On Mon, Jan 31, 2022 at 12:42 AM Lukasz Szaszkiewicz < ***@***.***> wrote: Hi @gracenng <https://github.com/gracenng>, it looks like it won't make it to 1.24. Please leave me some instructions on how to update the KEP so it won't show up on your query. — Reply to this email directly, view it on GitHub <#3157 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKCRLOZTCK54ONCDRDEXZCTUYZDP3ANCNFSM5MHDCLRQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Dingshujie]

@p0lyn0mial how is feature going? we met a lot of server overloading scene(kube-apiserver oom or master oom), looking forward to this feature. if you need help，you can assign some tasks to me.

[JohnRusk]

@Dingshujie BTW, if you have more than a few hundred nodes, then this fix will probably help to reduce API server load kubernetes/kubernetes#109381 . (If you have more than a few thousand nodes it will definitely help). It solves a bug where the kubelets would keep doing the same listing calls over and over again (typically immediately after a restart of the API server). Obvoiusly, other list issues are possible too, so that fix won't solve them all for you. But it might help.

[Dingshujie]

@Dingshujie BTW, if you have more than a few hundred nodes, then this fix will probably help to reduce API server load kubernetes/kubernetes#109381 . (If you have more than a few thousand nodes it will definitely help). It solves a bug where the kubelets would keep doing the same listing calls over and over again (typically immediately after a restart of the API server). Obvoiusly, other list issues are possible too, so that fix won't solve them all for you. But it might help.

@JohnRusk thanks, i already cherry-pick this to 1.23 branch. but this fixs can't slove the situation that custom-defined operator list issues.
and finally expect all list from watch cache

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[wojtek-t]

/remove-lifecycle rotten

[p0lyn0mial]

/milestone v1.26

[k8s-ci-robot]

@p0lyn0mial: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.26

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[p0lyn0mial]

Hey @jenshu, done ^.

[jenshu]

/milestone v1.32

[rashansmith]

Hello @p0lyn0mial 👋, v1.32 Comms team here,

Thank you for creating a placeholder PR for a feature blog!

The deadline for completing blog content to be Ready for Review is Monday 25th November 2024 . For more information about writing a blog see the blog contribution guidelines.

Please feel free to reach out if you have any questions!

[dipesh-rawat]

Hello 👋, 1.33 Enhancements Lead here.

I’m closing milestone 1.32 now. If you'd like to work on this enhancement in v1.33, please have the SIG lead opt-in by adding the lead-opted-in label, which ensures it gets added to the tracking board. Also, please set the milestone to v1.33 using /milestone v1.33.
Thanks!

/remove-label lead-opted-in
/milestone clear

[wojtek-t]

/label lead-opted-in
/milestone v1.33

[jenshu]

Hello @p0lyn0mial 👋, v1.33 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th February 2025 / 19:00 PDT Thursday 13th February 2025.

This enhancement is targeting stage beta for v1.33 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.33.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 6th February 2025 so that the PRR team has enough time to review your KEP.

For this KEP, we would just need to update the following:

• in the kep.yaml, the latest-milestone should be updated to v1.33

The status of this enhancement is marked as At risk for enhancements freeze. Please keep the issue description up-to-date with appropriate stages as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[dipesh-rawat]

Hi @p0lyn0mial 👋, 1.33 Enhancements team here,

Just a quick friendly reminder as we approach the enhancements freeze later this week, at 02:00 UTC Friday 14th February 2025 / 19:00 PDT Thursday 13th February 2025.

The current status of this enhancement is marked as At risk for enhancement freeze. There are a few requirements mentioned in the comment #3157 (comment) that still need to be completed.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[jenshu]

Hi @p0lyn0mial 👋, v1.33 Enhancements team here,

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

If you still wish to progress this enhancement in v1.33, please file an exception request as soon as possible, within three days. If you have any questions, you can reach out in the #release-enhancements channel on Slack and we'll be happy to help. Thanks!

/milestone clear