-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement RuntimeConfig CRI call #8722
Implement RuntimeConfig CRI call #8722
Conversation
Hi @marquiz. Thanks for your PR. I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Early peek to an implementation |
5715945
to
2163191
Compare
/cc @haircommander |
@marquiz: GitHub didn't allow me to request PR reviews from the following users: haircommander. Note that only containerd members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Required to support upcoming Kubernetes (v1.28) features. Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
2163191
to
276462d
Compare
Ready for review |
/ok-to-test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good! A couple comments / questions, request for a chat. Cheers
pkg/cri/server/runtime_config.go
Outdated
|
||
func (c *criService) getCgroupDriver(ctx context.Context) runtime.CgroupDriver { | ||
// Go through the runtime handlers in a predictable order, starting from the | ||
// default handler, others sorted in alphabetical order |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's have a conversation about defaulting... see if there isn't a better way than alphabetical, and / or see if there is an issue here:
Noting for the conversation that we probably need to consider adding validation against our config, and talk about other considerations if a runtime is requested that is not the same (different cgroup driver) than as reported via RuntimeConfig.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's have a conversation about defaulting... see if there isn't a better way than alphabetical, and / or see if there is an issue here:
Let's have a chat.
Noting for the conversation that we probably need to consider adding validation against our config, and talk about other considerations if a runtime is requested that is not the same (different cgroup driver) than as reported via RuntimeConfig.
Sounds pretty much like converging back to the old faithful "global" systemd_cgroup
setting albeit in a very complicated fashion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that might be the correct path until we can report said global and per/runtime ... and the validation would let admins know..
e6b7fbc
to
3fa3834
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to edit the config description...
https://github.com/containerd/containerd/blame/main/docs/cri/config.md#L13-L33
and possibly explain some down in the related runtime option fields for ShimCgroup and SystemdCgroup
pkg/cri/sbserver/runtime_config.go
Outdated
log.G(ctx).Debugf("runtime handler %q does not provide cgroup driver information", handler) | ||
} | ||
|
||
// If no runtime handlers have a setting, return systemd as the default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs to check whether systemd is running
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @AkihiroSuda, I think this is the right thing to do. And, now that I think of it, maybe we should just drop the SystemdCgroup
setting from the runc handler and just rely on auto-detection everywhere. Using cgroupfs
when system.slice
and folks are around feels like a mis-configuration to me 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When SystemdCgroup
is unspecified it can just detect the driver automatically.
Explicit configuration of SystemdCgroup
should be still supported for compatibility reason.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using cgroupfs when system.slice and folks are around feels like a mis-configuration to me 🤔
Still legit when the systemd unit is Delegate
-d.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still legit when the systemd unit is
Delegate
-d.
Hmm, that's true, in principle. But afaiu this scenario is not supported by kubelet at least. And, I think it's still a mis-configuration if the unit has Delegate=yes
but you have SystemdCgroup=true
(or vice versa if Delegate=off
but you'd have SystemdCgroup=false
. So, even in this scenario the right path would seem to be auto-detection (another level of auto-detection, i.e. auto-detection on the pod-sandbox unit level). WDYT @AkihiroSuda?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I implemented systemd autodetection in case no runc plugins are configured (i.e. we cannot get the setting from there).
Regarding whether SystemdCgroup
is explicitly set in the configuration, I don't think there's any way to detect that. It's a bool that's false by default so 🤷♀️
pkg/cri/sbserver/runtime_config.go
Outdated
func (c *criService) RuntimeConfig(ctx context.Context, r *runtime.RuntimeConfigRequest) (*runtime.RuntimeConfigResponse, error) { | ||
resp := &runtime.RuntimeConfigResponse{ | ||
Linux: &runtime.LinuxRuntimeConfiguration{ | ||
CgroupDriver: c.getCgroupDriver(ctx), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of topic: The kubelet takes the responsibility to create pod cgroup and then the CRI fills the container in it. I think that is reason why there is easy to cause mismatch. IMO, the end goal should be like that kubelet just lets the CRI to maintain the real lifecycle of container and pod. Currently, it's split-brain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I fully agree with this ❤️
3fa3834
to
0f21904
Compare
I updated the PR with auto-detection in case the driver cannot be determined from the runtime-handler options. Regarding ordering (in case the default runtime-handler doesn't provide the setting) I left the alphabetical ordering as there are no other/better suggestions. TBH, I think it should not be allowed to have multiple runc-based runtime classes with conflicting cgroup driver setting (different runtime classes having different PTAL @AkihiroSuda @mikebrow @fuweid |
0f21904
to
9302091
Compare
I made one more change: made the cgroup driver detection code Linux-only, other OS'es resorting to a stub |
The rpc only reports one field, i.e. the cgroup driver, to kubelet. Containerd determines the effective cgroup driver by looking at all runtime handlers, starting from the default runtime handler (the rest in alphabetical order), and returning the cgroup driver setting of the first runtime handler that supports one. If no runtime handler supports cgroup driver (i.e. has a config option for it) containerd falls back to auto-detection, returning systemd if systemd is running and cgroupfs otherwise. This patch implements the CRI server side of Kubernetes KEP-4033: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4033-group-driver-detection-over-cri Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Backporting to v1.6 is probably fine if it is necessary for Kube && it doesn't break compatibility for existing clusters
Thanks for the reviews @mikebrow @AkihiroSuda and @fuweid. I think the PR title "postfix" ( |
@marquiz missing dco signature on the docs commit.. |
Document the user visible changes related to the new RuntimeConfig RPC. Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
17ddeef
to
c62abcb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I'm going to remove the breaking label for now since there isn't enough explanation in the title to indicate what is breaking. It just appears as an addition. We can add a note to the 2.0 release notes explaining the cgroup change if there is user action required in some cases. |
> When determining the cgroup driver, containerd uses the `SystemdCgroup` | ||
> setting from runc-based runtime classes, starting from the default runtime | ||
> class. If no runc-based runtime classes have been configured containerd | ||
> relies on auto-detection based on determining if systemd is running. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the impact/breaking
label because this seems to break an existing node that is (mis)using non-systemd cgroup driver on systemd distro
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can add a note about it, it just looked confusing in the change log. It implies that the RuntimeConfig CRI call itself is a breaking change. #9340
We can clean up the language in future betas
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, maybe I have misread the paragraph
containerd 2.0.0-rc.5 Welcome to the v2.0.0-rc.5 release of containerd! *This is a pre-release of containerd* The first major release of containerd 2.x focuses on the continued stability of containerd's core feature set with an easy upgrade from containerd 1.x. This release includes the stabilization of new features added in the last 1.x release as well as the removal of features which were deprecated in 1.x. The goal is to support the vast community of containerd users well into the future along with their ever increasing deployment footprints and variety of use cases. * Add Update API for sandbox controller ([#9903](containerd/containerd#9903)) * Configure otel from env instead of config.toml ([#8970](containerd/containerd#8970)) * Enable NRI by default ([#9744](containerd/containerd#9744)) * Add PluginInfo to introspection API ([#9442](containerd/containerd#9442)) * Remove overlayfs volatile option on temp mounts ([#9555](containerd/containerd#9555)) * Expose usage of deprecated features ([#9258](containerd/containerd#9258)) * Use Intel ISA-L's igzip if available ([#9200](containerd/containerd#9200)) * Introduce top level config migration ([#9223](containerd/containerd#9223)) * Add image delete target ([#8989](containerd/containerd#8989)) * Remove `LimitNOFILE` from `containerd.service` ([#8924](containerd/containerd#8924)) * Add support for image expiration during garbage collection ([#9022](containerd/containerd#9022)) * Reduce the contention between ref lock and boltdb lock in content store ([#8792](containerd/containerd#8792)) * Remove "containerd.io/restart.logpath" label ([#8264](containerd/containerd#8264)) * Remove `aufs` snapshotter ([#8263](containerd/containerd#8263)) * Fix deadlock during NRI plugin registration ([containerd/nri#79](containerd/nri#79)) * Fix deadlock when writing to pipe blocks ([containerd/ttrpc#168](containerd/ttrpc#168)) * Generate attestation for artifacts during release ([#10543](containerd/containerd#10543)) * Use 'UserSpecifiedImage' from CRI to set the image-name annotation ([#10747](containerd/containerd#10747)) * Add support to set loopback to up ([#10238](containerd/containerd#10238)) * Add support for multiple subscribers to CRI container events ([#9661](containerd/containerd#9661)) * Enable CDI by default ([#9621](containerd/containerd#9621)) * Remove non-sandboxed CRI implementation ([#9228](containerd/containerd#9228)) * Add support for userns in stateless and stateful pods with idmap mounts (KEP-127, k8s >= 1.27) ([#8287](containerd/containerd#8287)) * Use sandboxed CRI by default ([#8994](containerd/containerd#8994)) * Implement RuntimeConfig CRI call ([#8722](containerd/containerd#8722)) * Add support for user namespaces (KEP-127) ([#8803](containerd/containerd#8803)) * Remove CRI v1alpha2 ([#8276](containerd/containerd#8276)) * Add api Go module and move all protos under api ([#10151](containerd/containerd#10151)) * Move packages based on contributing guide ([#9365](containerd/containerd#9365)) * Generalize plugin library ([#9214](containerd/containerd#9214)) * Use github.com/containerd/log ([#9086](containerd/containerd#9086)) * Support to syncfs after pull by using diff plugin ([#10284](containerd/containerd#10284)) * Skip "unknown" in image platform listing ([#10257](containerd/containerd#10257)) * Update unpacker to fetch all provided content ([#10202](containerd/containerd#10202)) * Enable Transfer service API to support plain HTTP ([#10024](containerd/containerd#10024)) * Enable Transfer service to use registry configuration directory ([#9908](containerd/containerd#9908)) * Disable the support for Schema 1 images ([#9765](containerd/containerd#9765)) * Update Transfer service to add OCI descriptors to Progress structure ([#9630](containerd/containerd#9630)) * Update import and export to allow references to missing content ([#9554](containerd/containerd#9554)) * Add option to perform syncfs after pull ([#9401](containerd/containerd#9401)) * Add image verifier transfer service plugin system based on a binary directory ([#8493](containerd/containerd#8493)) * Implement RuntimeStatus.features.supplemental_groups_policy from KEP-3619 ([#10410](containerd/containerd#10410)) * Add pprof to runc-shim ([#10242](containerd/containerd#10242)) * Provide runtime options in plugin info ([#10251](containerd/containerd#10251)) * Store bootstrap parameters in sandbox metadata ([#9736](containerd/containerd#9736)) * Update apparmor to allow confined runc to kill containers ([#10123](containerd/containerd#10123)) * Support vsock connection to task api ([#9738](containerd/containerd#9738)) * Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([#9320](containerd/containerd#9320)) * Switch runc shim to task service v3 and fix restore ([#9233](containerd/containerd#9233)) * Add sandboxer configuration and move sandbox controllers to plugins ([#8268](containerd/containerd#8268)) * Add annotations to CreateSandbox request ([#8960](containerd/containerd#8960)) * Add SandboxMetrics ([#8680](containerd/containerd#8680)) * Publish sandbox events ([#8602](containerd/containerd#8602)) * Remove the CriuPath field from runc's options ([#8279](containerd/containerd#8279)) * Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([#8262](containerd/containerd#8262)) * [medium] RAPL accessible to a container [GHSA-7ww5-4wqc-m92c](GHSA-7ww5-4wqc-m92c) * Remove `disable_cgroup` from CRI config ([#10594](containerd/containerd#10594)) * Disable the support for Schema 1 images ([#9765](containerd/containerd#9765)) * Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([#9320](containerd/containerd#9320)) * Move client to subpackage ([#9316](containerd/containerd#9316)) * Remove `LimitNOFILE` from `containerd.service` ([#8924](containerd/containerd#8924)) * Remove CRI v1alpha2 ([#8276](containerd/containerd#8276)) * Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([#8262](containerd/containerd#8262)) * Remove "containerd.io/restart.logpath" label ([#8264](containerd/containerd#8264)) * Remove `aufs` snapshotter ([#8263](containerd/containerd#8263)) * Update warnings for deprecated CRI config fields ([#10509](containerd/containerd#10509)) * Add type alias for event Envelope ([#10279](containerd/containerd#10279)) * Postpone removal of deprecated CRI config properties ([#9966](containerd/containerd#9966)) * Deprecate go-plugin configuration option ([#9238](containerd/containerd#9238)) * CNI conf_template in CRI is no longer deprecated ([#8637](containerd/containerd#8637)) Please try out the release binaries and report any issues at https://github.com/containerd/containerd/issues. * Derek McGowan * Akihiro Suda * Maksym Pavlenko * Wei Fu * Phil Estes * Sebastiaan van Stijn * Samuel Karp * Stefan Berger * Kazuyoshi Kato * Rodrigo Campos * Danny Canter * Abel Feng * Akhil Mohan * Kirtana Ashok * Gabriel Adrian Samfira * Austin Vazquez * Iceber Gu * Krisztian Litkey * Kohei Tokunaga * Mike Brown * Jin Dong * Bjorn Neergaard * Justin Chadwell * rongfu.leng * James Sturtevant * Davanum Srinivas * Paul "TBBle" Hampson * Henry Wang * Brian Goff * Enrico Weigelt * Laura Brehm * Marat Radchenko * Paweł Gronowski * Shingo Omura * Hsing-Yu (David) Chen * Ilya Hanov * Cardy.Tang * Swagat Bora * Aditi Sharma * Amit Barve * Bryant Biggs * Evan Lezar * James Jenkins * Jordan Liggitt * Kay Yan * Markus Lehtonen * Nashwan Azhari * Shuaiyi Zhang * Vinayak Goyal * helen * Alexandru Matei * Anthony Nandaa * Avi Deitcher * Charity Kathure * Cory Snider * Ed Bartosh * Etienne Champetier * Kevin Parsons * Michael Zappa * Milas Bowman * ningmingxiao * yanggang * zounengren * Aditya Ramani * Adrian Reber * Amir M. Ghazanfari * Artem Khramov * Brad Davidson * Chen Yiyang * Christian Muehlhaeuser * Djordje Lukic * Edgar Lee * Eric Lin * Ethan Lowman * Jiang Liu * June Rhodes * Kern Walster * Lucas Rattz * Mahamed Ali * Maksim An * Michael Crosby * Peteris Rudzusiks * Sam Edwards * Samruddhi Khandale * Sascha Grunert * Steve Griffith * Tony Fang * VERNOU Cédric * Vishal Reddy Gurrala * hang.jiang * harshitasao * jerryzhuang * lengrongfu * roman-kiselenko * zhanluxianshen * Aaron Lehmann * Adrien Delorme * Alex Couture-Beil * Alex Ellis * Alex Rodriguez * Angelos Kolaitis * Antonio Huete Jimenez * Arash Haghighat * Ben Foster * Bin Tang * Bin Xin * BinBin He * Brennan Kinney * Changqing Li * ChengenH * ChengyuZhu6 * Christian Stewart * Colin O'Dell * Craig Ingram * Daisy Rong * David Porter * Derek Nola * Eng Zer Jun * Erikson Tung * Fabiano Fidêncio * Fahed Dorgaa * Gary McDonald * Iain Macdonald * James Lakin * Jan Dubois * Jaroslav Jindrak * Javier Maestro * Jian Wang * Jiongchi Yu * Julien Balestra * Kir Kolyshkin * Kirill A. Korinsky * Konstantin Khlebnikov * Mauri de Souza Meneguzzo * Pan Yibo * Paul Meyer * Qasim Sarfraz * Qiutong Song * Reinhard Tartler * Robbie Buxton * Robert-André Mauchin * Ruihua Wen * Sameer * Shengjing Zhu * Shiming Zhang * Shukui Yang * Talon * Tariq Ibrahim * Tianon Gravi * Tim Hockin * TinaMor * Tobias Klauser * Tomáš Virtus * Tõnis Tiigi * Wang Xinwen * William Chen * Xinyang Ge * Yibo Zhuang * Yury Gargay * Zechun Chen * Zhang Tianyang * Zoe * baijia * bo.jiang * bzsuni * charles-chenzz * chschumacher1994 * guangli.bao * guangwu * jinda.ljd * krglosse * pigletfly * rokkiter * wangxiang * zhangpeng * zhaojizhuang * 吴小白 * 张钰 * 沈陵 * 谭九鼎 * **dario.cat/mergo** v1.0.1 **_new_** * **github.com/AdaLogics/go-fuzz-headers** 1f10f66a31bf -> ced1acdcaa24 * **github.com/AdamKorcz/go-118-fuzz-build** 5330a85ea652 -> 8075edf89bb0 * **github.com/Microsoft/go-winio** v0.6.0 -> v0.6.2 * **github.com/Microsoft/hcsshim** v0.10.0-rc.7 -> v0.12.6 * **github.com/cenkalti/backoff/v4** v4.2.0 -> v4.3.0 * **github.com/cespare/xxhash/v2** v2.2.0 -> v2.3.0 * **github.com/checkpoint-restore/checkpointctl** v1.2.1 **_new_** * **github.com/checkpoint-restore/go-criu/v7** v7.2.0 **_new_** * **github.com/cilium/ebpf** v0.9.1 -> v0.11.0 * **github.com/containerd/cgroups/v3** v3.0.1 -> v3.0.3 * **github.com/containerd/console** v1.0.3 -> v1.0.4 * **github.com/containerd/containerd/api** v1.8.0-rc.3 **_new_** * **github.com/containerd/continuity** v0.3.0 -> v0.4.3 * **github.com/containerd/errdefs** v0.1.0 **_new_** * **github.com/containerd/go-cni** v1.1.9 -> v1.1.10 * **github.com/containerd/go-runc** v1.0.0 -> v1.1.0 * **github.com/containerd/imgcrypt** v1.1.7 -> v1.2.0-rc1 * **github.com/containerd/log** v0.1.0 **_new_** * **github.com/containerd/nri** v0.3.0 -> v0.6.1 * **github.com/containerd/otelttrpc** ea5083fda723 **_new_** * **github.com/containerd/platforms** v0.2.1 **_new_** * **github.com/containerd/plugin** v0.1.0 **_new_** * **github.com/containerd/ttrpc** v1.2.1 -> v1.2.5 * **github.com/containerd/typeurl/v2** v2.1.0 -> v2.2.0 * **github.com/containernetworking/cni** v1.1.2 -> v1.2.3 * **github.com/containernetworking/plugins** v1.2.0 -> v1.5.1 * **github.com/containers/ocicrypt** v1.1.6 -> v1.2.0 * **github.com/cpuguy83/go-md2man/v2** v2.0.2 -> v2.0.4 * **github.com/davecgh/go-spew** v1.1.1 -> d8f796af33cc * **github.com/distribution/reference** v0.6.0 **_new_** * **github.com/emicklei/go-restful/v3** v3.10.1 -> v3.11.0 * **github.com/felixge/httpsnoop** v1.0.4 **_new_** * **github.com/fsnotify/fsnotify** v1.6.0 -> v1.7.0 * **github.com/fxamacker/cbor/v2** v2.7.0 **_new_** * **github.com/go-jose/go-jose/v4** v4.0.2 **_new_** * **github.com/go-logr/logr** v1.2.3 -> v1.4.2 * **github.com/golang/protobuf** v1.5.2 -> v1.5.4 * **github.com/google/go-cmp** v0.5.9 -> v0.6.0 * **github.com/google/uuid** v1.3.0 -> v1.6.0 * **github.com/gorilla/websocket** v1.5.0 **_new_** * **github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus** v1.0.1 **_new_** * **github.com/grpc-ecosystem/go-grpc-middleware/v2** v2.1.0 **_new_** * **github.com/grpc-ecosystem/grpc-gateway/v2** v2.7.0 -> v2.22.0 * **github.com/intel/goresctrl** v0.3.0 -> v0.7.0 * **github.com/klauspost/compress** v1.16.0 -> v1.17.10 * **github.com/mdlayher/socket** v0.4.1 **_new_** * **github.com/mdlayher/vsock** v1.2.1 **_new_** * **github.com/moby/spdystream** v0.2.0 -> v0.4.0 * **github.com/moby/sys/mountinfo** v0.6.2 -> v0.7.2 * **github.com/moby/sys/sequential** v0.5.0 -> v0.6.0 * **github.com/moby/sys/signal** v0.7.0 -> v0.7.1 * **github.com/moby/sys/symlink** v0.2.0 -> v0.3.0 * **github.com/moby/sys/user** v0.3.0 **_new_** * **github.com/moby/sys/userns** v0.1.0 **_new_** * **github.com/munnerz/goautoneg** a7dc8b61c822 **_new_** * **github.com/mxk/go-flowrate** cca7078d478f **_new_** * **github.com/opencontainers/image-spec** 3a7f492d3f1b -> v1.1.0 * **github.com/opencontainers/runtime-spec** v1.1.0-rc.1 -> v1.2.0 * **github.com/opencontainers/runtime-tools** 946c877fa809 -> 2e043c6bd626 * **github.com/pelletier/go-toml/v2** v2.2.3 **_new_** * **github.com/pmezard/go-difflib** v1.0.0 -> 5d4384ee4fb2 * **github.com/prometheus/client_golang** v1.14.0 -> v1.20.4 * **github.com/prometheus/client_model** v0.3.0 -> v0.6.1 * **github.com/prometheus/common** v0.37.0 -> v0.55.0 * **github.com/prometheus/procfs** v0.8.0 -> v0.15.1 * **github.com/sirupsen/logrus** v1.9.0 -> v1.9.3 * **github.com/stretchr/testify** v1.8.2 -> v1.9.0 * **github.com/urfave/cli/v2** v2.27.4 **_new_** * **github.com/vishvananda/netlink** v1.2.1-beta.2 -> v1.3.0 * **github.com/vishvananda/netns** 2eb08e3e575f -> v0.0.4 * **github.com/x448/float16** v0.8.4 **_new_** * **github.com/xrash/smetrics** 686a1a2994c1 **_new_** * **go.etcd.io/bbolt** v1.3.7 -> v1.3.11 * **go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc** v0.40.0 -> v0.55.0 * **go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp** v0.55.0 **_new_** * **go.opentelemetry.io/otel** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/metric** v0.37.0 -> v1.30.0 * **go.opentelemetry.io/otel/sdk** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/trace** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/proto/otlp** v0.19.0 -> v1.3.1 * **golang.org/x/crypto** v0.1.0 -> v0.27.0 * **golang.org/x/exp** aacd6d4b4611 **_new_** * **golang.org/x/mod** v0.7.0 -> v0.21.0 * **golang.org/x/net** v0.7.0 -> v0.29.0 * **golang.org/x/oauth2** v0.4.0 -> v0.22.0 * **golang.org/x/sync** v0.1.0 -> v0.8.0 * **golang.org/x/sys** v0.6.0 -> v0.25.0 * **golang.org/x/term** v0.5.0 -> v0.24.0 * **golang.org/x/text** v0.7.0 -> v0.18.0 * **golang.org/x/time** 90d013bbcef8 -> v0.3.0 * **google.golang.org/genproto/googleapis/api** 8af14fe29dc1 **_new_** * **google.golang.org/genproto/googleapis/rpc** 8af14fe29dc1 **_new_** * **google.golang.org/grpc** v1.53.0 -> v1.67.0 * **google.golang.org/protobuf** v1.28.1 -> v1.34.2 * **k8s.io/api** v0.26.2 -> v0.31.1 * **k8s.io/apimachinery** v0.26.2 -> v0.31.1 * **k8s.io/apiserver** v0.26.2 -> v0.31.1 * **k8s.io/client-go** v0.26.2 -> v0.31.1 * **k8s.io/component-base** v0.26.2 -> v0.31.1 * **k8s.io/cri-api** v0.26.2 -> v0.32.0-alpha.0 * **k8s.io/klog/v2** v2.90.1 -> v2.130.1 * **k8s.io/kubelet** v0.31.1 **_new_** * **k8s.io/utils** a5ecb0141aa5 -> 18e509b52bc8 * **sigs.k8s.io/json** f223a00ba0e2 -> bc3834ca7abd * **sigs.k8s.io/structured-merge-diff/v4** v4.2.3 -> v4.4.1 * **sigs.k8s.io/yaml** v1.3.0 -> v1.4.0 * **tags.cncf.io/container-device-interface** v0.8.0 **_new_** * **tags.cncf.io/container-device-interface/specs-go** v0.8.0 **_new_** Previous release can be found at [v1.7.0](https://github.com/containerd/containerd/releases/tag/v1.7.0) * `containerd-<VERSION>-<OS>-<ARCH>.tar.gz`: ✅Recommended. Dynamically linked with glibc 2.31 (Ubuntu 20.04). * `containerd-static-<VERSION>-<OS>-<ARCH>.tar.gz`: Statically linked. Expected to be used on non-glibc Linux distributions. Not position-independent. In addition to containerd, typically you will have to install [runc](https://github.com/opencontainers/runc/releases) and [CNI plugins](https://github.com/containernetworking/plugins/releases) from their official sites too. See also the [Getting Started](https://github.com/containerd/containerd/blob/main/docs/getting-started.md) documentation.
containerd 2.0.0-rc.5 Welcome to the v2.0.0-rc.5 release of containerd! *This is a pre-release of containerd* The first major release of containerd 2.x focuses on the continued stability of containerd's core feature set with an easy upgrade from containerd 1.x. This release includes the stabilization of new features added in the last 1.x release as well as the removal of features which were deprecated in 1.x. The goal is to support the vast community of containerd users well into the future along with their ever increasing deployment footprints and variety of use cases. * Add Update API for sandbox controller ([containerd#9903](containerd#9903)) * Configure otel from env instead of config.toml ([containerd#8970](containerd#8970)) * Enable NRI by default ([containerd#9744](containerd#9744)) * Add PluginInfo to introspection API ([containerd#9442](containerd#9442)) * Remove overlayfs volatile option on temp mounts ([containerd#9555](containerd#9555)) * Expose usage of deprecated features ([containerd#9258](containerd#9258)) * Use Intel ISA-L's igzip if available ([containerd#9200](containerd#9200)) * Introduce top level config migration ([containerd#9223](containerd#9223)) * Add image delete target ([containerd#8989](containerd#8989)) * Remove `LimitNOFILE` from `containerd.service` ([containerd#8924](containerd#8924)) * Add support for image expiration during garbage collection ([containerd#9022](containerd#9022)) * Reduce the contention between ref lock and boltdb lock in content store ([containerd#8792](containerd#8792)) * Remove "containerd.io/restart.logpath" label ([containerd#8264](containerd#8264)) * Remove `aufs` snapshotter ([containerd#8263](containerd#8263)) * Fix deadlock during NRI plugin registration ([containerd/nri#79](containerd/nri#79)) * Fix deadlock when writing to pipe blocks ([containerd/ttrpc#168](containerd/ttrpc#168)) * Generate attestation for artifacts during release ([containerd#10543](containerd#10543)) * Use 'UserSpecifiedImage' from CRI to set the image-name annotation ([containerd#10747](containerd#10747)) * Add support to set loopback to up ([containerd#10238](containerd#10238)) * Add support for multiple subscribers to CRI container events ([containerd#9661](containerd#9661)) * Enable CDI by default ([containerd#9621](containerd#9621)) * Remove non-sandboxed CRI implementation ([containerd#9228](containerd#9228)) * Add support for userns in stateless and stateful pods with idmap mounts (KEP-127, k8s >= 1.27) ([containerd#8287](containerd#8287)) * Use sandboxed CRI by default ([containerd#8994](containerd#8994)) * Implement RuntimeConfig CRI call ([containerd#8722](containerd#8722)) * Add support for user namespaces (KEP-127) ([containerd#8803](containerd#8803)) * Remove CRI v1alpha2 ([containerd#8276](containerd#8276)) * Add api Go module and move all protos under api ([containerd#10151](containerd#10151)) * Move packages based on contributing guide ([containerd#9365](containerd#9365)) * Generalize plugin library ([containerd#9214](containerd#9214)) * Use github.com/containerd/log ([containerd#9086](containerd#9086)) * Support to syncfs after pull by using diff plugin ([containerd#10284](containerd#10284)) * Skip "unknown" in image platform listing ([containerd#10257](containerd#10257)) * Update unpacker to fetch all provided content ([containerd#10202](containerd#10202)) * Enable Transfer service API to support plain HTTP ([containerd#10024](containerd#10024)) * Enable Transfer service to use registry configuration directory ([containerd#9908](containerd#9908)) * Disable the support for Schema 1 images ([containerd#9765](containerd#9765)) * Update Transfer service to add OCI descriptors to Progress structure ([containerd#9630](containerd#9630)) * Update import and export to allow references to missing content ([containerd#9554](containerd#9554)) * Add option to perform syncfs after pull ([containerd#9401](containerd#9401)) * Add image verifier transfer service plugin system based on a binary directory ([containerd#8493](containerd#8493)) * Implement RuntimeStatus.features.supplemental_groups_policy from KEP-3619 ([containerd#10410](containerd#10410)) * Add pprof to runc-shim ([containerd#10242](containerd#10242)) * Provide runtime options in plugin info ([containerd#10251](containerd#10251)) * Store bootstrap parameters in sandbox metadata ([containerd#9736](containerd#9736)) * Update apparmor to allow confined runc to kill containers ([containerd#10123](containerd#10123)) * Support vsock connection to task api ([containerd#9738](containerd#9738)) * Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([containerd#9320](containerd#9320)) * Switch runc shim to task service v3 and fix restore ([containerd#9233](containerd#9233)) * Add sandboxer configuration and move sandbox controllers to plugins ([containerd#8268](containerd#8268)) * Add annotations to CreateSandbox request ([containerd#8960](containerd#8960)) * Add SandboxMetrics ([containerd#8680](containerd#8680)) * Publish sandbox events ([containerd#8602](containerd#8602)) * Remove the CriuPath field from runc's options ([containerd#8279](containerd#8279)) * Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([containerd#8262](containerd#8262)) * [medium] RAPL accessible to a container [GHSA-7ww5-4wqc-m92c](GHSA-7ww5-4wqc-m92c) * Remove `disable_cgroup` from CRI config ([containerd#10594](containerd#10594)) * Disable the support for Schema 1 images ([containerd#9765](containerd#9765)) * Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([containerd#9320](containerd#9320)) * Move client to subpackage ([containerd#9316](containerd#9316)) * Remove `LimitNOFILE` from `containerd.service` ([containerd#8924](containerd#8924)) * Remove CRI v1alpha2 ([containerd#8276](containerd#8276)) * Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([containerd#8262](containerd#8262)) * Remove "containerd.io/restart.logpath" label ([containerd#8264](containerd#8264)) * Remove `aufs` snapshotter ([containerd#8263](containerd#8263)) * Update warnings for deprecated CRI config fields ([containerd#10509](containerd#10509)) * Add type alias for event Envelope ([containerd#10279](containerd#10279)) * Postpone removal of deprecated CRI config properties ([containerd#9966](containerd#9966)) * Deprecate go-plugin configuration option ([containerd#9238](containerd#9238)) * CNI conf_template in CRI is no longer deprecated ([containerd#8637](containerd#8637)) Please try out the release binaries and report any issues at https://github.com/containerd/containerd/issues. * Derek McGowan * Akihiro Suda * Maksym Pavlenko * Wei Fu * Phil Estes * Sebastiaan van Stijn * Samuel Karp * Stefan Berger * Kazuyoshi Kato * Rodrigo Campos * Danny Canter * Abel Feng * Akhil Mohan * Kirtana Ashok * Gabriel Adrian Samfira * Austin Vazquez * Iceber Gu * Krisztian Litkey * Kohei Tokunaga * Mike Brown * Jin Dong * Bjorn Neergaard * Justin Chadwell * rongfu.leng * James Sturtevant * Davanum Srinivas * Paul "TBBle" Hampson * Henry Wang * Brian Goff * Enrico Weigelt * Laura Brehm * Marat Radchenko * Paweł Gronowski * Shingo Omura * Hsing-Yu (David) Chen * Ilya Hanov * Cardy.Tang * Swagat Bora * Aditi Sharma * Amit Barve * Bryant Biggs * Evan Lezar * James Jenkins * Jordan Liggitt * Kay Yan * Markus Lehtonen * Nashwan Azhari * Shuaiyi Zhang * Vinayak Goyal * helen * Alexandru Matei * Anthony Nandaa * Avi Deitcher * Charity Kathure * Cory Snider * Ed Bartosh * Etienne Champetier * Kevin Parsons * Michael Zappa * Milas Bowman * ningmingxiao * yanggang * zounengren * Aditya Ramani * Adrian Reber * Amir M. Ghazanfari * Artem Khramov * Brad Davidson * Chen Yiyang * Christian Muehlhaeuser * Djordje Lukic * Edgar Lee * Eric Lin * Ethan Lowman * Jiang Liu * June Rhodes * Kern Walster * Lucas Rattz * Mahamed Ali * Maksim An * Michael Crosby * Peteris Rudzusiks * Sam Edwards * Samruddhi Khandale * Sascha Grunert * Steve Griffith * Tony Fang * VERNOU Cédric * Vishal Reddy Gurrala * hang.jiang * harshitasao * jerryzhuang * lengrongfu * roman-kiselenko * zhanluxianshen * Aaron Lehmann * Adrien Delorme * Alex Couture-Beil * Alex Ellis * Alex Rodriguez * Angelos Kolaitis * Antonio Huete Jimenez * Arash Haghighat * Ben Foster * Bin Tang * Bin Xin * BinBin He * Brennan Kinney * Changqing Li * ChengenH * ChengyuZhu6 * Christian Stewart * Colin O'Dell * Craig Ingram * Daisy Rong * David Porter * Derek Nola * Eng Zer Jun * Erikson Tung * Fabiano Fidêncio * Fahed Dorgaa * Gary McDonald * Iain Macdonald * James Lakin * Jan Dubois * Jaroslav Jindrak * Javier Maestro * Jian Wang * Jiongchi Yu * Julien Balestra * Kir Kolyshkin * Kirill A. Korinsky * Konstantin Khlebnikov * Mauri de Souza Meneguzzo * Pan Yibo * Paul Meyer * Qasim Sarfraz * Qiutong Song * Reinhard Tartler * Robbie Buxton * Robert-André Mauchin * Ruihua Wen * Sameer * Shengjing Zhu * Shiming Zhang * Shukui Yang * Talon * Tariq Ibrahim * Tianon Gravi * Tim Hockin * TinaMor * Tobias Klauser * Tomáš Virtus * Tõnis Tiigi * Wang Xinwen * William Chen * Xinyang Ge * Yibo Zhuang * Yury Gargay * Zechun Chen * Zhang Tianyang * Zoe * baijia * bo.jiang * bzsuni * charles-chenzz * chschumacher1994 * guangli.bao * guangwu * jinda.ljd * krglosse * pigletfly * rokkiter * wangxiang * zhangpeng * zhaojizhuang * 吴小白 * 张钰 * 沈陵 * 谭九鼎 * **dario.cat/mergo** v1.0.1 **_new_** * **github.com/AdaLogics/go-fuzz-headers** 1f10f66a31bf -> ced1acdcaa24 * **github.com/AdamKorcz/go-118-fuzz-build** 5330a85ea652 -> 8075edf89bb0 * **github.com/Microsoft/go-winio** v0.6.0 -> v0.6.2 * **github.com/Microsoft/hcsshim** v0.10.0-rc.7 -> v0.12.6 * **github.com/cenkalti/backoff/v4** v4.2.0 -> v4.3.0 * **github.com/cespare/xxhash/v2** v2.2.0 -> v2.3.0 * **github.com/checkpoint-restore/checkpointctl** v1.2.1 **_new_** * **github.com/checkpoint-restore/go-criu/v7** v7.2.0 **_new_** * **github.com/cilium/ebpf** v0.9.1 -> v0.11.0 * **github.com/containerd/cgroups/v3** v3.0.1 -> v3.0.3 * **github.com/containerd/console** v1.0.3 -> v1.0.4 * **github.com/containerd/containerd/api** v1.8.0-rc.3 **_new_** * **github.com/containerd/continuity** v0.3.0 -> v0.4.3 * **github.com/containerd/errdefs** v0.1.0 **_new_** * **github.com/containerd/go-cni** v1.1.9 -> v1.1.10 * **github.com/containerd/go-runc** v1.0.0 -> v1.1.0 * **github.com/containerd/imgcrypt** v1.1.7 -> v1.2.0-rc1 * **github.com/containerd/log** v0.1.0 **_new_** * **github.com/containerd/nri** v0.3.0 -> v0.6.1 * **github.com/containerd/otelttrpc** ea5083fda723 **_new_** * **github.com/containerd/platforms** v0.2.1 **_new_** * **github.com/containerd/plugin** v0.1.0 **_new_** * **github.com/containerd/ttrpc** v1.2.1 -> v1.2.5 * **github.com/containerd/typeurl/v2** v2.1.0 -> v2.2.0 * **github.com/containernetworking/cni** v1.1.2 -> v1.2.3 * **github.com/containernetworking/plugins** v1.2.0 -> v1.5.1 * **github.com/containers/ocicrypt** v1.1.6 -> v1.2.0 * **github.com/cpuguy83/go-md2man/v2** v2.0.2 -> v2.0.4 * **github.com/davecgh/go-spew** v1.1.1 -> d8f796af33cc * **github.com/distribution/reference** v0.6.0 **_new_** * **github.com/emicklei/go-restful/v3** v3.10.1 -> v3.11.0 * **github.com/felixge/httpsnoop** v1.0.4 **_new_** * **github.com/fsnotify/fsnotify** v1.6.0 -> v1.7.0 * **github.com/fxamacker/cbor/v2** v2.7.0 **_new_** * **github.com/go-jose/go-jose/v4** v4.0.2 **_new_** * **github.com/go-logr/logr** v1.2.3 -> v1.4.2 * **github.com/golang/protobuf** v1.5.2 -> v1.5.4 * **github.com/google/go-cmp** v0.5.9 -> v0.6.0 * **github.com/google/uuid** v1.3.0 -> v1.6.0 * **github.com/gorilla/websocket** v1.5.0 **_new_** * **github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus** v1.0.1 **_new_** * **github.com/grpc-ecosystem/go-grpc-middleware/v2** v2.1.0 **_new_** * **github.com/grpc-ecosystem/grpc-gateway/v2** v2.7.0 -> v2.22.0 * **github.com/intel/goresctrl** v0.3.0 -> v0.7.0 * **github.com/klauspost/compress** v1.16.0 -> v1.17.10 * **github.com/mdlayher/socket** v0.4.1 **_new_** * **github.com/mdlayher/vsock** v1.2.1 **_new_** * **github.com/moby/spdystream** v0.2.0 -> v0.4.0 * **github.com/moby/sys/mountinfo** v0.6.2 -> v0.7.2 * **github.com/moby/sys/sequential** v0.5.0 -> v0.6.0 * **github.com/moby/sys/signal** v0.7.0 -> v0.7.1 * **github.com/moby/sys/symlink** v0.2.0 -> v0.3.0 * **github.com/moby/sys/user** v0.3.0 **_new_** * **github.com/moby/sys/userns** v0.1.0 **_new_** * **github.com/munnerz/goautoneg** a7dc8b61c822 **_new_** * **github.com/mxk/go-flowrate** cca7078d478f **_new_** * **github.com/opencontainers/image-spec** 3a7f492d3f1b -> v1.1.0 * **github.com/opencontainers/runtime-spec** v1.1.0-rc.1 -> v1.2.0 * **github.com/opencontainers/runtime-tools** 946c877fa809 -> 2e043c6bd626 * **github.com/pelletier/go-toml/v2** v2.2.3 **_new_** * **github.com/pmezard/go-difflib** v1.0.0 -> 5d4384ee4fb2 * **github.com/prometheus/client_golang** v1.14.0 -> v1.20.4 * **github.com/prometheus/client_model** v0.3.0 -> v0.6.1 * **github.com/prometheus/common** v0.37.0 -> v0.55.0 * **github.com/prometheus/procfs** v0.8.0 -> v0.15.1 * **github.com/sirupsen/logrus** v1.9.0 -> v1.9.3 * **github.com/stretchr/testify** v1.8.2 -> v1.9.0 * **github.com/urfave/cli/v2** v2.27.4 **_new_** * **github.com/vishvananda/netlink** v1.2.1-beta.2 -> v1.3.0 * **github.com/vishvananda/netns** 2eb08e3e575f -> v0.0.4 * **github.com/x448/float16** v0.8.4 **_new_** * **github.com/xrash/smetrics** 686a1a2994c1 **_new_** * **go.etcd.io/bbolt** v1.3.7 -> v1.3.11 * **go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc** v0.40.0 -> v0.55.0 * **go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp** v0.55.0 **_new_** * **go.opentelemetry.io/otel** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/metric** v0.37.0 -> v1.30.0 * **go.opentelemetry.io/otel/sdk** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/otel/trace** v1.14.0 -> v1.30.0 * **go.opentelemetry.io/proto/otlp** v0.19.0 -> v1.3.1 * **golang.org/x/crypto** v0.1.0 -> v0.27.0 * **golang.org/x/exp** aacd6d4b4611 **_new_** * **golang.org/x/mod** v0.7.0 -> v0.21.0 * **golang.org/x/net** v0.7.0 -> v0.29.0 * **golang.org/x/oauth2** v0.4.0 -> v0.22.0 * **golang.org/x/sync** v0.1.0 -> v0.8.0 * **golang.org/x/sys** v0.6.0 -> v0.25.0 * **golang.org/x/term** v0.5.0 -> v0.24.0 * **golang.org/x/text** v0.7.0 -> v0.18.0 * **golang.org/x/time** 90d013bbcef8 -> v0.3.0 * **google.golang.org/genproto/googleapis/api** 8af14fe29dc1 **_new_** * **google.golang.org/genproto/googleapis/rpc** 8af14fe29dc1 **_new_** * **google.golang.org/grpc** v1.53.0 -> v1.67.0 * **google.golang.org/protobuf** v1.28.1 -> v1.34.2 * **k8s.io/api** v0.26.2 -> v0.31.1 * **k8s.io/apimachinery** v0.26.2 -> v0.31.1 * **k8s.io/apiserver** v0.26.2 -> v0.31.1 * **k8s.io/client-go** v0.26.2 -> v0.31.1 * **k8s.io/component-base** v0.26.2 -> v0.31.1 * **k8s.io/cri-api** v0.26.2 -> v0.32.0-alpha.0 * **k8s.io/klog/v2** v2.90.1 -> v2.130.1 * **k8s.io/kubelet** v0.31.1 **_new_** * **k8s.io/utils** a5ecb0141aa5 -> 18e509b52bc8 * **sigs.k8s.io/json** f223a00ba0e2 -> bc3834ca7abd * **sigs.k8s.io/structured-merge-diff/v4** v4.2.3 -> v4.4.1 * **sigs.k8s.io/yaml** v1.3.0 -> v1.4.0 * **tags.cncf.io/container-device-interface** v0.8.0 **_new_** * **tags.cncf.io/container-device-interface/specs-go** v0.8.0 **_new_** Previous release can be found at [v1.7.0](https://github.com/containerd/containerd/releases/tag/v1.7.0) * `containerd-<VERSION>-<OS>-<ARCH>.tar.gz`: ✅Recommended. Dynamically linked with glibc 2.31 (Ubuntu 20.04). * `containerd-static-<VERSION>-<OS>-<ARCH>.tar.gz`: Statically linked. Expected to be used on non-glibc Linux distributions. Not position-independent. In addition to containerd, typically you will have to install [runc](https://github.com/opencontainers/runc/releases) and [CNI plugins](https://github.com/containernetworking/plugins/releases) from their official sites too. See also the [Getting Started](https://github.com/containerd/containerd/blob/main/docs/getting-started.md) documentation.
The rpc only reports one field, i.e. the cgroup driver, to kubelet.
Containerd determines the effective cgroup driver by looking at all
runtime handlers, starting from the default runtime handler (the rest in
alphabetical order), and returning the cgroup driver setting of the
first runtime handler that supports one. If no runtime handler supports
cgroup driver (i.e. has a config option for it) systemd is returned as a
fallback default.
This patch implements the CRI server side of Kubernetes KEP-4033:
https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4033-group-driver-detection-over-cri
Refs kubernetes/kubernetes#118770