Pod Scheduling Readiness

[Huang-Wei]

Enhancement Description

• One-line enhancement description (can be used as a release note): Add a schedulingGates field to the Pod spec that marks a Pod's scheduling readiness.
• Kubernetes Enhancement Proposal: 3521-pod-scheduling-readiness/README.md
• Discussion Link: https://docs.google.com/document/d/1cPxF3wIbGRw_inv4CCG6z6qC4WjHAPR5GLT__dRBx7I/edit#
• Primary contact (assignee): @Huang-Wei
• Responsible SIGs: sig-scheduling
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.26
  • Beta release target (x.y): 1.27
  • Stable release target (x.y): 1.29

Alpha

Beta Give feedback

1. KEP: Pod Scheduling Readiness #3522
   approved cncf-cla: yes kind/kep lgtm sig/scheduling size/XXL tide/merge-method-squash +7
   
2. [KEP-3521] Part 1: New Pod API .spec.schedulingGates kubernetes#113274
   api-review approved area/code-generation area/e2e-test-framework area/test cncf-cla: yes kind/api-change kind/feature lgtm needs-priority release-note sig/apps sig/scheduling sig/testing size/XL triage/accepted +16
   
3. [KEP-3521] Part 2: Core scheduling implementation kubernetes#113275
   api-review approved area/code-generation area/e2e-test-framework area/stable-metrics area/test cncf-cla: yes kind/api-change kind/feature lgtm needs-priority release-note sig/api-machinery sig/apps sig/instrumentation sig/scheduling sig/testing size/XL triage/accepted +19
   

   
4. [KEP-3521] Part 3: Bug fixes, integration & E2E Test kubernetes#113442
   approved area/code-generation area/e2e-test-framework area/stable-metrics area/test cncf-cla: yes kind/api-change kind/feature lgtm needs-priority release-note-none sig/api-machinery sig/apps sig/instrumentation sig/scheduling sig/testing size/L triage/accepted +18
   

   
5. Fix an accuracy issue of scheduler_pending_pods metric kubernetes#113946
   approved cncf-cla: yes kind/bug lgtm needs-priority needs-triage release-note-none sig/scheduling size/L +9
   
6. Doc for Alpha feature PodSchedulingReadiness website#37675
   approved cncf-cla: yes language/en lgtm sig/docs sig/scheduling size/L +7
   

Beta

Beta Give feedback

1. KEP-3521: Graduate PodSchedulingReadiness to beta #3739
   approved cncf-cla: yes kind/kep lgtm sig/scheduling size/L +6
   
2. Umbrella issue tracking Beta impl. of KEP 3521 (Pod Scheduling Readiness) kubernetes#113608
   3 of 4
   lifecycle/stale needs-triage sig/scheduling +3
   
3. Doc for Beta feature PodSchedulingReadiness website#39773
   approved cncf-cla: yes language/en lgtm sig/docs size/XS +6
   

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[Huang-Wei]

/sig scheduling

[ahg-g]

/label lead-opted-in

[ahg-g]

/milestone v1.26

[Atharva-Shinde]

Hey @kerthcet 👋, 1.26 Enhancements team here!

Just checking in as we approach Enhancements Freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable
• KEP has an updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would need to:

• Change the status of kep.yaml to implementable (I've also added a comment review)
• Include the new updated PR of this KEP in the Issue Description and get it merged before Enhancements Freeze to make this enhancement eligible for 1.26 release.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well.
Thank you :)

[Huang-Wei]

Thanks @Atharva-Shinde.

• The KEP is updated to marked as implementable
• Waiting for @johnbelamaric or @wojtek-t to be the approver of PRR: KEP: Pod Scheduling Readiness #3522 (comment)

[Atharva-Shinde]

Hello @Huang-Wei 👋, just a quick check-in again, as we approach the 1.26 Enhancements freeze.

Please plan to get the PR #3522 merged before Enhancements freeze on 18:00 PDT on Thursday 6th October 2022 i.e tomorrow

For note, the current status of the enhancement is marked at-risk :)

[Huang-Wei]

Thanks for the reminder. It's 99% accomplished atm, just some final comments waiting for the approver to +1.

[rhockenbury]

With #3522 merged, we have this marked as tracked for v1.26.

[marosset]

Hi @Huang-Wei 👋,

Checking in once more as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• [KEP-3521] Part 1: New Pod API .spec.schedulingGates kubernetes#113274
• [KEP-3521] Part 2: Core scheduling implementation kubernetes#113275
• [KEP-3521] Part 3: Bug fixes, integration & E2E Test kubernetes#113442

Please ensure all of these PRs are linked to this issue as well and mentioned in the initial issue description.

As always, we are here to help should questions come up. Thanks!

[Huang-Wei]

@marosset yes, those 3 PRs are code implementation for this KEP in alpha stage. I just updated the issue description to get them linked.

[krol3]

Hello @Huang-Wei 👋, 1.26 Release Docs Lead here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9. Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Any doubt, reach us! Thank you!

[marosset]

Hi @Huang-Wei 👋,

Checking in once more as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• [KEP-3521] Part 2: Core scheduling implementation kubernetes#113275
• [KEP-3521] Part 3: Bug fixes, integration & E2E Test kubernetes#113442

As always, we are here to help should questions come up. Thanks!

[Huang-Wei]

@marosset ACK, I'm working with reviewers to get 2 pending PRs merged by tomorrow.

[Huang-Wei]

I'm working with reviewers to get 2 pending PRs merged by tomorrow.

All dev work has been merged.

[marosset]

/remove-label lead-opted-in
/remove-label tracked/yes
/label tracked/no
/milestone clear

[alculquicondor]

Not sure if I should post this here or #2926, but it would be nice if scheduling gates are mutable in the job template.

[ahg-g]

Not sure if I should post this here or #2926, but it would be nice if scheduling gates are mutable in the job template.

Part of the KEP update for mutable scheduling directives already: #3709

[Huang-Wei]

/label lead-opted-in

[sftim]

If we graduate this to beta, we'll need to update https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/ and maybe other Pod pages, to mention / link to the details about it.

[ahg-g]

/stage beta
/milestone v1.27
/label lead-opted-in

[npolshakova]

Hello @Huang-Wei 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT Thursday 9th February 2023.

This enhancement is targeting for stage beta for 1.27 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.27
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

It looks like #3739 will address these issues.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[npolshakova]

Looks like with #3739 merged this enhancement as tracked for v1.27.
Thanks!

/remove-label tracked/no
/label tracked/yes

[npolshakova]

Hi @Huang-Wei,

Checking in as we approach 1.27 code freeze at 17:00 PDT on Tuesday 14th March 2023.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• add(scheduler): implement "plugin_execution_duration_seconds" metric in PreEnqueue kubernetes#116201

Please let me know if there are any other PRs in k/k I should be tracking for this KEP.
As always, we are here to help should questions come up. Thanks!

[Huang-Wei]

@npolshakova I will have my eye on kubernetes/kubernetes#116201 to ensure it's completed before code freeze.

[pacoxu]

https://kubernetes.io/docs/concepts/workloads/pods/#pod-update-and-replacement
a document update here may be needed for spec.schedulingGates that is updatable.