KEP-4033: graduate to GA #5370

marquiz · 2025-06-04T12:03:44Z

One-line PR description: graduate KEP-4033 to GA

Issue link: Discover cgroup driver from CRI #4033

Other comments:

marquiz · 2025-06-04T12:04:03Z

/cc @haircommander @mrunalp @mikebrow

keps/sig-node/4033-group-driver-detection-over-cri/README.md

SergeyKanzhelev · 2025-06-05T00:06:13Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

-<!--
-Describe the consequences on existing workloads (e.g., if this is a runtime
-feature, can it break the existing applications?).
+In GA, the fallback behavior (and the kubelet `--cgroup-driver` flag) is


we require a few releases before removing the flag as per deprecation policy. Please update. (The flag can do nothing, it just should be around marked as deprecated)

There are more than a few places within our own repos that are still setting this flag
https://cs.k8s.io/?q=cgroup(.)%3Fdriver&i=nope&literal=nope&files=&excludeFiles=&repos=

And then users may be opting to configure it indirectly, e.g. through kubeadm or kops configurations.

Check, I updated the PR to align with the official deprecation policy.

Now changed so that the fallback behavior is kept as long as the cmdline flag and config option.

BenTheElder · 2025-06-05T16:39:05Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

 message, e.g.:

 ```
 cgroupDriver option has been deprecated and will be dropped in a future release. Please upgrade to a CRI implementation that supports cgroup-driver detection.
 ```

+The `--cgroup-driver` flag and the cgroupDriver configuration option will be
+deprecated and have no effect when support for the feature is graduated to GA.


What's the motivation behind making CRI auto-detection a hard requirement and ignoring explicit user configuration (which would enable using older runtimes for example?), the flag doesn't seem like a lot of code to retain.

+1 same question. seems to add risk with little benefit?

Now changed so that the fallback behavior is kept as long as the cmdline flag and config option. These will be marked as deprecated and will eventually be removed as per the K8s deprecation policy.

johnbelamaric · 2025-06-06T22:46:55Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

-flag). Further, the kubeletConfig field and `--cgroup-driver` flag will be
-marked as deprecated, to be dropped when support for the feature is adopted by
-CRI-O and containerd. Usage of the deprecated setting will produce a log
+flag). For beta, the kubeletConfig field and `--cgroup-driver` flag will be


"will be" or "have been"?

They were not truly deprecated in beta (except for the log message saying/lying so). Now changed the wording to

In beta, resorting to the fallback behavior will produce a log message like:

johnbelamaric · 2025-06-06T22:48:28Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

 message, e.g.:

 ```
 cgroupDriver option has been deprecated and will be dropped in a future release. Please upgrade to a CRI implementation that supports cgroup-driver detection.
 ```

+The `--cgroup-driver` flag and the cgroupDriver configuration option will be
+deprecated and have no effect when support for the feature is graduated to GA.


+1 same question. seems to add risk with little benefit?

johnbelamaric · 2025-06-06T22:50:25Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

-You can take a look at one potential example of such test in:
-https://github.com/kubernetes/kubernetes/pull/97058/files#diff-7826f7adbc1996a05ab52e3f5f02429e94b68ce6bce0dc534d1be636154fded3R246-R282
-->
-


presumably the line below can now reference those tests?

The removed block was a comment block and not related to this KEP or feature.

johnbelamaric · 2025-06-06T22:52:03Z

keps/sig-node/4033-group-driver-detection-over-cri/README.md

-checking if there are objects with field X set) may be a last resort. Avoid
-logs or events for this purpose.
-->
-
 Kubelet and container runtime version. The


if my nodes are going NotReady, how can I tell that it is due to this feature, without having to log in to all 10,000 nodes?

This should not now happen as long as the fallback behavior is in place.

After the deprecated functionality is removed this will be similar to other kubelet startup errors. The node will be set to NotReady by the node-controller (visible in node events). When the kubelet does not start I don't know what other methods you have other than examine kubelet logs (log in or via scraped logs)

Thoughts @haircommander ?

could maybe have an event emitted but that's not all that reliable

Propose to retain fallback behavior in GA, but, mark the the --cgroup-driver flag and cgroupDriver config option as deprecated and eventually drop the according to the Kubernetes deprecation policy.

marquiz · 2025-06-11T18:39:41Z

Updated: Changed to proposal to retain fallback behavior in GA, but, mark the the --cgroup-driver flag and the CgroupDriver config option as deprecated and eventually drop the according to the Kubernetes deprecation policy.

SergeyKanzhelev

/lgtm
/approve

haircommander · 2025-06-16T15:41:51Z

/lgtm

Signed-off-by: Peter Hunt <pehunt@redhat.com>

deads2k · 2025-06-16T17:09:37Z

a few questions

How will the sig decide on when the right time is?
How will the kubelet warn cluster-admins that their CRIs are too old or incompatible before hard-failing in an unrecoverable way?

These are similar in spirit to the thread here: #5370 (comment) and I think we can do much better than, "oh, the kubelet exits". For instance, in 1.34, you could add a metric indicating that the CRI is too old for a future version.

haircommander · 2025-06-16T17:11:51Z

/reopen

not sure why it closed

k8s-ci-robot · 2025-06-16T17:11:56Z

@haircommander: Failed to re-open PR: state cannot be changed. There are no new commits on the marquiz:devel/kep-cgroup-driver branch.

In response to this:

/reopen

not sure why it closed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

haircommander · 2025-06-16T17:13:05Z

/reopen

k8s-ci-robot · 2025-06-16T17:13:12Z

@haircommander: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

haircommander · 2025-06-16T18:46:02Z

I added a metric cri_losing_support to give admins a hint that there will be a future version that will be incompatible. I think this + documentation will give admins warning enough. I also clarified we're dropping support for containerd 1.7 in k8s 1.36, which corresponds with containerd 1.7 losing upstream support. WDYT @deads2k

…d 1.7 will be dropped in 1.36 Signed-off-by: Peter Hunt <pehunt@redhat.com>

deads2k · 2025-06-17T18:11:44Z

I added a metric cri_losing_support to give admins a hint that there will be a future version that will be incompatible. I think this + documentation will give admins warning enough. I also clarified we're dropping support for containerd 1.7 in k8s 1.36, which corresponds with containerd 1.7 losing upstream support. WDYT @deads2k

I think that addresses both John and my concerns. Thanks.

PRR lgtm
/approve

SergeyKanzhelev · 2025-06-18T16:06:07Z

/assign @mrunalp

as official approver for the KEP

k8s-ci-robot · 2025-06-18T16:08:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, marquiz, mrunalp, SergeyKanzhelev

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~keps/prod-readiness/OWNERS~~ [deads2k]
~~keps/sig-node/OWNERS~~ [mrunalp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

SergeyKanzhelev · 2025-06-18T16:24:46Z

/lgtm

marquiz added 3 commits June 4, 2025 12:11

KEP-4033: sync KEP with implementation

020f1d0

KEP-4033: update for GA

91267f2

KEP-4033: drop comment blocks from README.md

72e7a4c

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/node Categorizes an issue or PR as relevant to SIG Node. labels Jun 4, 2025

k8s-ci-robot requested review from johnbelamaric and mrunalp June 4, 2025 12:03

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 4, 2025

k8s-ci-robot requested review from haircommander and mikebrow June 4, 2025 12:04

marquiz mentioned this pull request Jun 4, 2025

Discover cgroup driver from CRI #4033

Open

13 tasks

haircommander reviewed Jun 4, 2025

View reviewed changes

keps/sig-node/4033-group-driver-detection-over-cri/README.md Show resolved Hide resolved

SergeyKanzhelev reviewed Jun 5, 2025

View reviewed changes

KEP-4033: align to kubernetes deprecation policy

a0ac8f1

BenTheElder reviewed Jun 5, 2025

View reviewed changes

johnbelamaric reviewed Jun 6, 2025

View reviewed changes

KEP-4033: update GA graduation

5c040c3

Propose to retain fallback behavior in GA, but, mark the the --cgroup-driver flag and cgroupDriver config option as deprecated and eventually drop the according to the Kubernetes deprecation policy.

SergeyKanzhelev approved these changes Jun 11, 2025

View reviewed changes

k8s-ci-robot assigned SergeyKanzhelev Jun 11, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 11, 2025

k8s-ci-robot assigned haircommander Jun 16, 2025

4033: replace PRR approver to deads2k

2895ae2

Signed-off-by: Peter Hunt <pehunt@redhat.com>

haircommander closed this Jun 16, 2025

haircommander force-pushed the devel/kep-cgroup-driver branch from 5c040c3 to caadc0f Compare June 16, 2025 17:06

k8s-ci-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 16, 2025

k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jun 16, 2025

k8s-ci-robot reopened this Jun 16, 2025

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 16, 2025

4033: add cri_losing_support metric and clarify support for container…

c783802

…d 1.7 will be dropped in 1.36 Signed-off-by: Peter Hunt <pehunt@redhat.com>

haircommander force-pushed the devel/kep-cgroup-driver branch from 15db405 to c783802 Compare June 16, 2025 18:50

k8s-ci-robot assigned mrunalp Jun 18, 2025

mrunalp approved these changes Jun 18, 2025

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 18, 2025

k8s-ci-robot merged commit a0f8483 into kubernetes:master Jun 18, 2025
4 checks passed

k8s-ci-robot added this to the v1.34 milestone Jun 18, 2025

KEP-4033: graduate to GA #5370

KEP-4033: graduate to GA #5370

Uh oh!

Conversation

marquiz commented Jun 4, 2025

Uh oh!

marquiz commented Jun 4, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BenTheElder Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marquiz commented Jun 11, 2025

Uh oh!

SergeyKanzhelev left a comment

Choose a reason for hiding this comment

Uh oh!

haircommander commented Jun 16, 2025

Uh oh!

deads2k commented Jun 16, 2025

Uh oh!

haircommander commented Jun 16, 2025

Uh oh!

k8s-ci-robot commented Jun 16, 2025

Uh oh!

haircommander commented Jun 16, 2025

Uh oh!

k8s-ci-robot commented Jun 16, 2025

Uh oh!

haircommander commented Jun 16, 2025

Uh oh!

deads2k commented Jun 17, 2025

Uh oh!

SergeyKanzhelev commented Jun 18, 2025

Uh oh!

k8s-ci-robot commented Jun 18, 2025

Uh oh!

SergeyKanzhelev commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

BenTheElder Jun 5, 2025 •

edited

Loading