Add configurable grace period to probes

[ehashman]

Enhancement Description

• One-line enhancement description (can be used as a release note): Add configurable gracePeriodSeconds field to liveness probes
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2238-liveness-probe-grace-period
• Discussion Link: terminationGracePeriodSeconds is overloaded kubernetes#64715 (comment)
• Primary contact (assignee): @rphillips
• Responsible SIGs: SIG Node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.21
  • Beta release target (x.y): 1.22
  • Stable release target (x.y): 1.27
• Alpha
  • KEP (k/enhancements) update PR(s):
    • Configurable probe termination grace periods #2241
  • Code (k/k) update PR(s):
    • Add Probe-level terminationGracePeriodSeconds kubernetes#99375
  • Docs (k/website) update PR(s):
    • Add documentation for Liveness Probe Grace Periods website#27084
• Beta
  • KEP (k/enhancements) update PR(s):
    • Update probe grace period KEP to beta #2638
  • Code (k/k) update PR(s):
    • Fix TerminationGracePeriodSeconds is negative (part 1) kubernetes#98866
    • Fix pkg/api/pod/util tests to ensure feature gate is set kubernetes#103420
    • Code changes for Probe-level Termination Grace Period Beta kubernetes#103168
    • Add validation for Prober TerminationGracePeriodSeconds kubernetes#103245
    • feat: set ProbeTerminationGracePeriod default to true kubernetes#108541
    • Add unit tests for grace period in killContainer func kubernetes#110733
  • Docs (k/website) update(s):
    • KEP-2238 probe-level grace period changes for beta/1.22 website#28930
    • [dev-1.25] Update feature gate docs for probe-level termination grace period website#34665

/sig node

[ehashman]

/milestone v1.21

[arunmk]

@ehashman I am one of the enhancement shadows for v1.21 and am reviewing this KEP. We have the enhancements freeze coming up on 2/9. Could you mention if PR #2241 be merged by then?

I looked at the PR and things seem ok from a KEP standpoint with the exception of the empty Graduation Criteria section. We will be needing that also to be present (for the current alpha phase at least) at the enhancements freeze deadline.

Thanks!

[arunmk]

Hi @ehashman

Enhancements Freeze is 1 day away, Feb 9th EOD PST

Enhancements team is aware that KEP update is currently in progress (PR #2241). I do not see a graduation criteria section in the PR and will need some details. The KEP looks good.

Any enhancements that do not complete the following requirements by the freeze will require an exception.

Assuming that the PR is merged, this would be the state of the KEP:
[DONE] The KEP must be merged in an implementable state
[DONE] The KEP must have test plans
[DONE] The KEP must have graduation criteria: this is missing in the PR
[DONE] The KEP must have a production readiness review

EDIT: updated with new commit in the PR. Note that the PR still needs to be merged by the enhancements freeze.

[ehashman]

Done; this was sort of implicit and I have added it explicitly.

[arunmk]

Thanks @ehashman. I will update the earlier status inline. (I'm not sure how rigorously the template is to be followed, but it is useful to follow it for the first few iterations so that we know what issues would arise).

[arunmk]

Hi @ehashman,

Since your Enhancement is scheduled to be in 1.21, please keep in mind the important upcoming dates:

• Tuesday, March 9th: Week 9 - Code Freeze
• Tuesday, March 16th: Week 10 - Docs Placeholder PR deadline
  • If this enhancement requires new docs or modification to existing docs, please follow the steps in the Open a placeholder PR doc to open a PR against k/website repo.

As a reminder, please link all of your k/k PR(s) and k/website PR(s) to this issue so we can track them.

Thanks!

[ehashman]

Thanks @arunmk, I was hoping to have some code out for this last week but didn't get a chance to work on it. Hoping today or tomorrow...

[arunmk]

Hi @ehashman ,

Enhancements team is currently tracking the following PR

• Add Probe-level terminationGracePeriodSeconds kubernetes#99375

I went through the review comments and there are a few comments requiring some decisions and there is also an api-review initiated. This is the only PR but it is a fairly large one at 15 KLOC. Since there are a few decision points and some test related comments, and also since the code freeze is one week away at 3/9, I am tagging this as At Risk for now.

Please do let me know if you think otherwise and the tag can be revised.

Thanks!

[arunmk]

Hi @ehashman,

A friendly reminder that Code freeze is 4 days away, March 9th EOD PST

Any enhancements that are NOT code complete by the freeze will be removed from the milestone and will require an exception to be added back.

Please also keep in mind that if this enhancement requires new docs or modification to existing docs, you'll need to follow the steps in the Open a placeholder PR doc to open a PR against k/website repo by March 16th EOD PST

Thanks!

[annajung]

Hi @ehashman, with code freeze now in effect, we are removing this enhancement from 1.21 release due to kubernetes/kubernetes#99375 not being merged or approved.

Feel free to file an exception to add this back into the release. thanks!

[ehashman]

Hi @annajung,

This was basically approved, but the API reviewer didn't do a /approve to make it Official. I just finished rebasing the PR and I will have reviewers take another look today. Will you still need me to send a full exception email? Landing this PR (+ docs) is the only requirement for alpha graduation.

[ehashman]

Filed exception request: https://groups.google.com/g/kubernetes-sig-node/c/fg6Gd8m9jHc/m/u6KOEZYwAAAJ

[annajung]

Hi @ehashman, with kubernetes/kubernetes#99375 merged in, we're marking this as code complete for the 1.21 release. Thank you

[ehashman]

/milestone v1.22

[bobbypage]

This needs lead opted in label for 1.27.

@SergeyKanzhelev @mrunalp can you please add?

[SergeyKanzhelev]

/label lead-opted-in

[SergeyKanzhelev]

/stage stable

[npolshakova]

Hello @rphillips 👋, 1.27 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT Thursday 9th February 2023.

This enhancement is targeting for stage stable for 1.27 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.27
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

With all the KEP requirements in place and merged into k/enhancements, this enhancement is all good for the upcoming enhancements freeze. 🚀

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[johnbelamaric]

@rphillips If it's moving from beta to stable it will need an updated PRR review, I don't see that.

[jeremyrickard]

I've unchecked the PRR review complete on the checklist above @npolshakova, this needs an updated PRR review.

[npolshakova]

Thanks, I'll update the enhancement tracking!

[johnbelamaric]

@rphillips please try to get a PR up soon, time is running short :)

[rphillips]

@johnbelamaric I have a PR over here #3830

[johnbelamaric]

thanks, sorry I didn't see it. this LGTM but I have to wait for sig approval before giving mine

[SergeyKanzhelev]

@npolshakova all requirements are satisfied now. Can you please mark this one as tracked?

[rphillips]

/assign @npolshakova

[npolshakova]

Great, with #3830 merged I'm marking this enhancement as tracked for v1.27.
Thanks!

/remove-label tracked/no
/label tracked/yes

[npolshakova]

Hi @rphillips,

Checking in as we approach 1.27 code freeze at 17:00 PDT on Tuesday 14th March 2023.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.
  • Fix TerminationGracePeriodSeconds is negative (part 1) kubernetes#98866
  • Fix pkg/api/pod/util tests to ensure feature gate is set kubernetes#103420
  • Code changes for Probe-level Termination Grace Period Beta kubernetes#103168
  • Add validation for Prober TerminationGracePeriodSeconds kubernetes#103245
  • feat: set ProbeTerminationGracePeriod default to true kubernetes#108541
  • Add unit tests for grace period in killContainer func kubernetes#110733

Please let me know if there are any other PRs in k/k I should be tracking for this KEP.
As always, we are here to help should questions come up. Thanks!

[Rishit-dagli]

Hi @ehashman 👋, I’m reaching out from the 1.27 Release Docs team. This enhancement is marked as ‘Needs Docs’ for the 1.27 release. If I understand correctly this does not have docs PR for 1.27, right?

If so, please follow the steps detailed in the documentation to open a PR against dev-1.27 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by March 16. For more information, please take a look at Documenting for a release to familiarize yourself with the documentation requirements for the release.
Please feel free to reach out with any questions. Thanks!

[mickeyboxell]

@rphillips @ehashman The deadline for getting docs into the 1.27 release is coming up quickly on April 4th. Please open a docs PR against dev-1.27 branch in the k/website repo ASAP.

[rphillips]

@mickeyboxell Thank you for the ping... Added a PR.

[SergeyKanzhelev]

/milestone v1.28