Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix deadlock during NRI plugin registration #79

Merged
merged 1 commit into from
Apr 18, 2024
Merged

Fix deadlock during NRI plugin registration #79

merged 1 commit into from
Apr 18, 2024

Conversation

bobbypage
Copy link
Contributor

During NRI external plugin registration:

  • acceptPluginConnections() is called
  • adaptation lock from nri/pkg/adaptation is acquired
  • syncFn is invoked
  • syncFn acquires NRI lock in pkg/nri/nri.go

During container lifecycle events such as ContainerStart

  • NRI lock is acquired in pkg/nri.go
  • adaptation lock is acquired in StateChange() in nri/pkg/adaptation

As a result, the locking order during NRI plugin registration is:

  • adaptation lock -> NRI lock

While the locking order during container starts is:

  • NRI lock -> adaptation lock

Due the fact that the locking order is inverted and not consistent, it it possible to encounter a deadlock.

To fix the issue, during NRI plugin registration, first acquire the NRI lock (done via syncFn call) and only after acquire the adaptation lock. This ensures that NRI plugin registration the locking order is adaption lock -> NRI lock, which is consistent with the locking order during container lifecycle events.

Fixes containerd/containerd#10085

@bobbypage
Copy link
Contributor Author

bobbypage commented Apr 18, 2024 •
edited
Loading

cc @klihub @acurtiz @samuelkarp

@bobbypage bobbypage mentioned this pull request Apr 18, 2024
Closed
@samuelkarp
Copy link
Member

WIP: Fix deadlock during NRI plugin registration

What are you looking for to make this non-WIP?

@klihub
Copy link
Member

klihub commented Apr 18, 2024

I think this should be fine.

@bobbypage
Fix deadlock during NRI plugin registration
c4893c7 
During NRI external plugin registration:

* `acceptPluginConnections()` is called
* adaptation lock from `nri/pkg/adaptation` is acquired
* `syncFn` is invoked
* `syncFn` acquires NRI lock in `pkg/nri/nri.go`

During container lifecycle events such as `ContainerStart`
* NRI lock is acquired in pkg/nri.go
* adaptation lock is acquired in `StateChange()` in `nri/pkg/adaptation`

As a result, the locking order during NRI plugin registration is:
* adaptation lock -> NRI lock

While the locking order during container starts is:
* NRI lock -> adaptation lock

Due the fact that the locking order is inverted and not consistent, it
it possible to encounter a deadlock.

To fix the issue, during NRI plugin registration, first acquire the NRI
lock (done via `syncFn` call) and only after acquire the adaptation lock.
This ensures that NRI plugin registration the locking order is adaption
lock -> NRI lock, which is consistent with the locking order during
container lifecycle events.

Fixes containerd/containerd#10085

Signed-off-by: David Porter <porterdavid@google.com>
@bobbypage bobbypage changed the title WIP: Fix deadlock during NRI plugin registration Fix deadlock during NRI plugin registration Apr 18, 2024
@bobbypage
Copy link
Contributor Author

WIP: Fix deadlock during NRI plugin registration

What are you looking for to make this non-WIP?

I removed WIP, wanted to test it again a few times against the repro in containerd/containerd#10085 (comment)

After a few runs of those steps, I can confirm that without this patch the deadlock always repros and with this patch I am no longer able to repro the deadlock.

@estesp estesp merged commit fa64d11 into containerd:main Apr 18, 2024
8 checks passed
@kiashok kiashok mentioned this pull request Apr 24, 2024
Merged
@dmcgowan dmcgowan mentioned this pull request May 17, 2024
Merged
@dmcgowan dmcgowan mentioned this pull request Jun 13, 2024
Merged
@ben-childs-docusign ben-childs-docusign mentioned this pull request Jul 22, 2024
Closed
@SergeyKanzhelev SergeyKanzhelev mentioned this pull request Aug 12, 2024
Open
@AkihiroSuda AkihiroSuda mentioned this pull request Aug 15, 2024
Closed
@dmcgowan dmcgowan mentioned this pull request Aug 27, 2024
Merged
@dmcgowan dmcgowan mentioned this pull request Sep 30, 2024
Merged
Mengkzhaoyun pushed a commit to open-beagle/containerd that referenced this pull request Oct 11, 2024
Merge tag 'v2.0.0-rc.5' into release-v2.0
c737fe2 
containerd 2.0.0-rc.5

Welcome to the v2.0.0-rc.5 release of containerd!
*This is a pre-release of containerd*

The first major release of containerd 2.x focuses on the continued stability of
containerd's core feature set with an easy upgrade from containerd 1.x. This
release includes the stabilization of new features added in the last 1.x release
as well as the removal of features which were deprecated in 1.x. The goal is to
support the vast community of containerd users well into the future along with
their ever increasing deployment footprints and variety of use cases.

* Add Update API for sandbox controller ([#9903](containerd/containerd#9903))
* Configure otel from env instead of config.toml ([#8970](containerd/containerd#8970))
* Enable NRI by default ([#9744](containerd/containerd#9744))
* Add PluginInfo to introspection API ([#9442](containerd/containerd#9442))
* Remove overlayfs volatile option on temp mounts ([#9555](containerd/containerd#9555))
* Expose usage of deprecated features ([#9258](containerd/containerd#9258))
* Use Intel ISA-L's igzip if available ([#9200](containerd/containerd#9200))
* Introduce top level config migration ([#9223](containerd/containerd#9223))
* Add image delete target ([#8989](containerd/containerd#8989))
* Remove `LimitNOFILE` from `containerd.service` ([#8924](containerd/containerd#8924))
* Add support for image expiration during garbage collection ([#9022](containerd/containerd#9022))
* Reduce the contention between ref lock and boltdb lock in content store ([#8792](containerd/containerd#8792))
* Remove "containerd.io/restart.logpath" label ([#8264](containerd/containerd#8264))
* Remove `aufs` snapshotter ([#8263](containerd/containerd#8263))
* Fix deadlock during NRI plugin registration ([containerd/nri#79](containerd/nri#79))
* Fix deadlock when writing to pipe blocks ([containerd/ttrpc#168](containerd/ttrpc#168))

* Generate attestation for artifacts during release ([#10543](containerd/containerd#10543))

* Use 'UserSpecifiedImage' from CRI to set the image-name annotation ([#10747](containerd/containerd#10747))
* Add support to set loopback to up ([#10238](containerd/containerd#10238))
* Add support for multiple subscribers to CRI container events ([#9661](containerd/containerd#9661))
* Enable CDI by default ([#9621](containerd/containerd#9621))
* Remove non-sandboxed CRI implementation ([#9228](containerd/containerd#9228))
* Add support for userns in stateless and stateful pods with idmap mounts (KEP-127, k8s >= 1.27) ([#8287](containerd/containerd#8287))
* Use sandboxed CRI by default ([#8994](containerd/containerd#8994))
* Implement RuntimeConfig CRI call ([#8722](containerd/containerd#8722))
* Add support for user namespaces (KEP-127) ([#8803](containerd/containerd#8803))
* Remove CRI v1alpha2 ([#8276](containerd/containerd#8276))

* Add api Go module and move all protos under api ([#10151](containerd/containerd#10151))
* Move packages based on contributing guide ([#9365](containerd/containerd#9365))
* Generalize plugin library ([#9214](containerd/containerd#9214))
* Use github.com/containerd/log ([#9086](containerd/containerd#9086))

* Support to syncfs after pull by using diff plugin ([#10284](containerd/containerd#10284))
* Skip "unknown" in image platform listing ([#10257](containerd/containerd#10257))
* Update unpacker to fetch all provided content ([#10202](containerd/containerd#10202))
* Enable Transfer service API to support plain HTTP ([#10024](containerd/containerd#10024))
* Enable Transfer service to use registry configuration directory ([#9908](containerd/containerd#9908))
* Disable the support for Schema 1 images ([#9765](containerd/containerd#9765))
* Update Transfer service to add OCI descriptors to Progress structure ([#9630](containerd/containerd#9630))
* Update import and export to allow references to missing content  ([#9554](containerd/containerd#9554))
* Add option to perform syncfs after pull ([#9401](containerd/containerd#9401))
* Add image verifier transfer service plugin system based on a binary directory ([#8493](containerd/containerd#8493))

* Implement  RuntimeStatus.features.supplemental_groups_policy from KEP-3619 ([#10410](containerd/containerd#10410))
* Add pprof to runc-shim ([#10242](containerd/containerd#10242))
* Provide runtime options in plugin info ([#10251](containerd/containerd#10251))
* Store bootstrap parameters in sandbox metadata ([#9736](containerd/containerd#9736))
* Update apparmor to allow confined runc to kill containers ([#10123](containerd/containerd#10123))
* Support vsock connection to task api ([#9738](containerd/containerd#9738))
* Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([#9320](containerd/containerd#9320))
* Switch runc shim to task service v3 and fix restore ([#9233](containerd/containerd#9233))
* Add sandboxer configuration and move sandbox controllers to plugins ([#8268](containerd/containerd#8268))
* Add annotations to CreateSandbox request ([#8960](containerd/containerd#8960))
* Add SandboxMetrics ([#8680](containerd/containerd#8680))
* Publish sandbox events ([#8602](containerd/containerd#8602))
* Remove the CriuPath field from runc's options ([#8279](containerd/containerd#8279))
* Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([#8262](containerd/containerd#8262))

* [medium] RAPL accessible to a container [GHSA-7ww5-4wqc-m92c](GHSA-7ww5-4wqc-m92c)

* Remove `disable_cgroup` from CRI config ([#10594](containerd/containerd#10594))
* Disable the support for Schema 1 images ([#9765](containerd/containerd#9765))
* Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([#9320](containerd/containerd#9320))
* Move client to subpackage ([#9316](containerd/containerd#9316))
* Remove `LimitNOFILE` from `containerd.service` ([#8924](containerd/containerd#8924))
* Remove CRI v1alpha2 ([#8276](containerd/containerd#8276))
* Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([#8262](containerd/containerd#8262))
* Remove "containerd.io/restart.logpath" label ([#8264](containerd/containerd#8264))
* Remove `aufs` snapshotter ([#8263](containerd/containerd#8263))

* Update warnings for deprecated CRI config fields ([#10509](containerd/containerd#10509))
* Add type alias for event Envelope ([#10279](containerd/containerd#10279))
* Postpone removal of deprecated CRI config properties ([#9966](containerd/containerd#9966))
* Deprecate go-plugin configuration option ([#9238](containerd/containerd#9238))
* CNI conf_template in CRI is no longer deprecated ([#8637](containerd/containerd#8637))

Please try out the release binaries and report any issues at
https://github.com/containerd/containerd/issues.

* Derek McGowan
* Akihiro Suda
* Maksym Pavlenko
* Wei Fu
* Phil Estes
* Sebastiaan van Stijn
* Samuel Karp
* Stefan Berger
* Kazuyoshi Kato
* Rodrigo Campos
* Danny Canter
* Abel Feng
* Akhil Mohan
* Kirtana Ashok
* Gabriel Adrian Samfira
* Austin Vazquez
* Iceber Gu
* Krisztian Litkey
* Kohei Tokunaga
* Mike Brown
* Jin Dong
* Bjorn Neergaard
* Justin Chadwell
* rongfu.leng
* James Sturtevant
* Davanum Srinivas
* Paul "TBBle" Hampson
* Henry Wang
* Brian Goff
* Enrico Weigelt
* Laura Brehm
* Marat Radchenko
* Paweł Gronowski
* Shingo Omura
* Hsing-Yu (David) Chen
* Ilya Hanov
* Cardy.Tang
* Swagat Bora
* Aditi Sharma
* Amit Barve
* Bryant Biggs
* Evan Lezar
* James Jenkins
* Jordan Liggitt
* Kay Yan
* Markus Lehtonen
* Nashwan Azhari
* Shuaiyi Zhang
* Vinayak Goyal
* helen
* Alexandru Matei
* Anthony Nandaa
* Avi Deitcher
* Charity Kathure
* Cory Snider
* Ed Bartosh
* Etienne Champetier
* Kevin Parsons
* Michael Zappa
* Milas Bowman
* ningmingxiao
* yanggang
* zounengren
* Aditya Ramani
* Adrian Reber
* Amir M. Ghazanfari
* Artem Khramov
* Brad Davidson
* Chen Yiyang
* Christian Muehlhaeuser
* Djordje Lukic
* Edgar Lee
* Eric Lin
* Ethan Lowman
* Jiang Liu
* June Rhodes
* Kern Walster
* Lucas Rattz
* Mahamed Ali
* Maksim An
* Michael Crosby
* Peteris Rudzusiks
* Sam Edwards
* Samruddhi Khandale
* Sascha Grunert
* Steve Griffith
* Tony Fang
* VERNOU Cédric
* Vishal Reddy Gurrala
* hang.jiang
* harshitasao
* jerryzhuang
* lengrongfu
* roman-kiselenko
* zhanluxianshen
* Aaron Lehmann
* Adrien Delorme
* Alex Couture-Beil
* Alex Ellis
* Alex Rodriguez
* Angelos Kolaitis
* Antonio Huete Jimenez
* Arash Haghighat
* Ben Foster
* Bin Tang
* Bin Xin
* BinBin He
* Brennan Kinney
* Changqing Li
* ChengenH
* ChengyuZhu6
* Christian Stewart
* Colin O'Dell
* Craig Ingram
* Daisy Rong
* David Porter
* Derek Nola
* Eng Zer Jun
* Erikson Tung
* Fabiano Fidêncio
* Fahed Dorgaa
* Gary McDonald
* Iain Macdonald
* James Lakin
* Jan Dubois
* Jaroslav Jindrak
* Javier Maestro
* Jian Wang
* Jiongchi Yu
* Julien Balestra
* Kir Kolyshkin
* Kirill A. Korinsky
* Konstantin Khlebnikov
* Mauri de Souza Meneguzzo
* Pan Yibo
* Paul Meyer
* Qasim Sarfraz
* Qiutong Song
* Reinhard Tartler
* Robbie Buxton
* Robert-André Mauchin
* Ruihua Wen
* Sameer
* Shengjing Zhu
* Shiming Zhang
* Shukui Yang
* Talon
* Tariq Ibrahim
* Tianon Gravi
* Tim Hockin
* TinaMor
* Tobias Klauser
* Tomáš Virtus
* Tõnis Tiigi
* Wang Xinwen
* William Chen
* Xinyang Ge
* Yibo Zhuang
* Yury Gargay
* Zechun Chen
* Zhang Tianyang
* Zoe
* baijia
* bo.jiang
* bzsuni
* charles-chenzz
* chschumacher1994
* guangli.bao
* guangwu
* jinda.ljd
* krglosse
* pigletfly
* rokkiter
* wangxiang
* zhangpeng
* zhaojizhuang
* 吴小白
* 张钰
* 沈陵
* 谭九鼎

* **dario.cat/mergo**                                                              v1.0.1 **_new_**
* **github.com/AdaLogics/go-fuzz-headers**                                         1f10f66a31bf -> ced1acdcaa24
* **github.com/AdamKorcz/go-118-fuzz-build**                                       5330a85ea652 -> 8075edf89bb0
* **github.com/Microsoft/go-winio**                                                v0.6.0 -> v0.6.2
* **github.com/Microsoft/hcsshim**                                                 v0.10.0-rc.7 -> v0.12.6
* **github.com/cenkalti/backoff/v4**                                               v4.2.0 -> v4.3.0
* **github.com/cespare/xxhash/v2**                                                 v2.2.0 -> v2.3.0
* **github.com/checkpoint-restore/checkpointctl**                                  v1.2.1 **_new_**
* **github.com/checkpoint-restore/go-criu/v7**                                     v7.2.0 **_new_**
* **github.com/cilium/ebpf**                                                       v0.9.1 -> v0.11.0
* **github.com/containerd/cgroups/v3**                                             v3.0.1 -> v3.0.3
* **github.com/containerd/console**                                                v1.0.3 -> v1.0.4
* **github.com/containerd/containerd/api**                                         v1.8.0-rc.3 **_new_**
* **github.com/containerd/continuity**                                             v0.3.0 -> v0.4.3
* **github.com/containerd/errdefs**                                                v0.1.0 **_new_**
* **github.com/containerd/go-cni**                                                 v1.1.9 -> v1.1.10
* **github.com/containerd/go-runc**                                                v1.0.0 -> v1.1.0
* **github.com/containerd/imgcrypt**                                               v1.1.7 -> v1.2.0-rc1
* **github.com/containerd/log**                                                    v0.1.0 **_new_**
* **github.com/containerd/nri**                                                    v0.3.0 -> v0.6.1
* **github.com/containerd/otelttrpc**                                              ea5083fda723 **_new_**
* **github.com/containerd/platforms**                                              v0.2.1 **_new_**
* **github.com/containerd/plugin**                                                 v0.1.0 **_new_**
* **github.com/containerd/ttrpc**                                                  v1.2.1 -> v1.2.5
* **github.com/containerd/typeurl/v2**                                             v2.1.0 -> v2.2.0
* **github.com/containernetworking/cni**                                           v1.1.2 -> v1.2.3
* **github.com/containernetworking/plugins**                                       v1.2.0 -> v1.5.1
* **github.com/containers/ocicrypt**                                               v1.1.6 -> v1.2.0
* **github.com/cpuguy83/go-md2man/v2**                                             v2.0.2 -> v2.0.4
* **github.com/davecgh/go-spew**                                                   v1.1.1 -> d8f796af33cc
* **github.com/distribution/reference**                                            v0.6.0 **_new_**
* **github.com/emicklei/go-restful/v3**                                            v3.10.1 -> v3.11.0
* **github.com/felixge/httpsnoop**                                                 v1.0.4 **_new_**
* **github.com/fsnotify/fsnotify**                                                 v1.6.0 -> v1.7.0
* **github.com/fxamacker/cbor/v2**                                                 v2.7.0 **_new_**
* **github.com/go-jose/go-jose/v4**                                                v4.0.2 **_new_**
* **github.com/go-logr/logr**                                                      v1.2.3 -> v1.4.2
* **github.com/golang/protobuf**                                                   v1.5.2 -> v1.5.4
* **github.com/google/go-cmp**                                                     v0.5.9 -> v0.6.0
* **github.com/google/uuid**                                                       v1.3.0 -> v1.6.0
* **github.com/gorilla/websocket**                                                 v1.5.0 **_new_**
* **github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus**            v1.0.1 **_new_**
* **github.com/grpc-ecosystem/go-grpc-middleware/v2**                              v2.1.0 **_new_**
* **github.com/grpc-ecosystem/grpc-gateway/v2**                                    v2.7.0 -> v2.22.0
* **github.com/intel/goresctrl**                                                   v0.3.0 -> v0.7.0
* **github.com/klauspost/compress**                                                v1.16.0 -> v1.17.10
* **github.com/mdlayher/socket**                                                   v0.4.1 **_new_**
* **github.com/mdlayher/vsock**                                                    v1.2.1 **_new_**
* **github.com/moby/spdystream**                                                   v0.2.0 -> v0.4.0
* **github.com/moby/sys/mountinfo**                                                v0.6.2 -> v0.7.2
* **github.com/moby/sys/sequential**                                               v0.5.0 -> v0.6.0
* **github.com/moby/sys/signal**                                                   v0.7.0 -> v0.7.1
* **github.com/moby/sys/symlink**                                                  v0.2.0 -> v0.3.0
* **github.com/moby/sys/user**                                                     v0.3.0 **_new_**
* **github.com/moby/sys/userns**                                                   v0.1.0 **_new_**
* **github.com/munnerz/goautoneg**                                                 a7dc8b61c822 **_new_**
* **github.com/mxk/go-flowrate**                                                   cca7078d478f **_new_**
* **github.com/opencontainers/image-spec**                                         3a7f492d3f1b -> v1.1.0
* **github.com/opencontainers/runtime-spec**                                       v1.1.0-rc.1 -> v1.2.0
* **github.com/opencontainers/runtime-tools**                                      946c877fa809 -> 2e043c6bd626
* **github.com/pelletier/go-toml/v2**                                              v2.2.3 **_new_**
* **github.com/pmezard/go-difflib**                                                v1.0.0 -> 5d4384ee4fb2
* **github.com/prometheus/client_golang**                                          v1.14.0 -> v1.20.4
* **github.com/prometheus/client_model**                                           v0.3.0 -> v0.6.1
* **github.com/prometheus/common**                                                 v0.37.0 -> v0.55.0
* **github.com/prometheus/procfs**                                                 v0.8.0 -> v0.15.1
* **github.com/sirupsen/logrus**                                                   v1.9.0 -> v1.9.3
* **github.com/stretchr/testify**                                                  v1.8.2 -> v1.9.0
* **github.com/urfave/cli/v2**                                                     v2.27.4 **_new_**
* **github.com/vishvananda/netlink**                                               v1.2.1-beta.2 -> v1.3.0
* **github.com/vishvananda/netns**                                                 2eb08e3e575f -> v0.0.4
* **github.com/x448/float16**                                                      v0.8.4 **_new_**
* **github.com/xrash/smetrics**                                                    686a1a2994c1 **_new_**
* **go.etcd.io/bbolt**                                                             v1.3.7 -> v1.3.11
* **go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc**  v0.40.0 -> v0.55.0
* **go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp**                v0.55.0 **_new_**
* **go.opentelemetry.io/otel**                                                     v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace**                            v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc**              v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp**              v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/metric**                                              v0.37.0 -> v1.30.0
* **go.opentelemetry.io/otel/sdk**                                                 v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/trace**                                               v1.14.0 -> v1.30.0
* **go.opentelemetry.io/proto/otlp**                                               v0.19.0 -> v1.3.1
* **golang.org/x/crypto**                                                          v0.1.0 -> v0.27.0
* **golang.org/x/exp**                                                             aacd6d4b4611 **_new_**
* **golang.org/x/mod**                                                             v0.7.0 -> v0.21.0
* **golang.org/x/net**                                                             v0.7.0 -> v0.29.0
* **golang.org/x/oauth2**                                                          v0.4.0 -> v0.22.0
* **golang.org/x/sync**                                                            v0.1.0 -> v0.8.0
* **golang.org/x/sys**                                                             v0.6.0 -> v0.25.0
* **golang.org/x/term**                                                            v0.5.0 -> v0.24.0
* **golang.org/x/text**                                                            v0.7.0 -> v0.18.0
* **golang.org/x/time**                                                            90d013bbcef8 -> v0.3.0
* **google.golang.org/genproto/googleapis/api**                                    8af14fe29dc1 **_new_**
* **google.golang.org/genproto/googleapis/rpc**                                    8af14fe29dc1 **_new_**
* **google.golang.org/grpc**                                                       v1.53.0 -> v1.67.0
* **google.golang.org/protobuf**                                                   v1.28.1 -> v1.34.2
* **k8s.io/api**                                                                   v0.26.2 -> v0.31.1
* **k8s.io/apimachinery**                                                          v0.26.2 -> v0.31.1
* **k8s.io/apiserver**                                                             v0.26.2 -> v0.31.1
* **k8s.io/client-go**                                                             v0.26.2 -> v0.31.1
* **k8s.io/component-base**                                                        v0.26.2 -> v0.31.1
* **k8s.io/cri-api**                                                               v0.26.2 -> v0.32.0-alpha.0
* **k8s.io/klog/v2**                                                               v2.90.1 -> v2.130.1
* **k8s.io/kubelet**                                                               v0.31.1 **_new_**
* **k8s.io/utils**                                                                 a5ecb0141aa5 -> 18e509b52bc8
* **sigs.k8s.io/json**                                                             f223a00ba0e2 -> bc3834ca7abd
* **sigs.k8s.io/structured-merge-diff/v4**                                         v4.2.3 -> v4.4.1
* **sigs.k8s.io/yaml**                                                             v1.3.0 -> v1.4.0
* **tags.cncf.io/container-device-interface**                                      v0.8.0 **_new_**
* **tags.cncf.io/container-device-interface/specs-go**                             v0.8.0 **_new_**

Previous release can be found at [v1.7.0](https://github.com/containerd/containerd/releases/tag/v1.7.0)
* `containerd-<VERSION>-<OS>-<ARCH>.tar.gz`:         ✅Recommended. Dynamically linked with glibc 2.31 (Ubuntu 20.04).
* `containerd-static-<VERSION>-<OS>-<ARCH>.tar.gz`:  Statically linked. Expected to be used on non-glibc Linux distributions. Not position-independent.

In addition to containerd, typically you will have to install [runc](https://github.com/opencontainers/runc/releases)
and [CNI plugins](https://github.com/containernetworking/plugins/releases) from their official sites too.

See also the [Getting Started](https://github.com/containerd/containerd/blob/main/docs/getting-started.md) documentation.
kiashok added a commit to kiashok/containerd-containerd that referenced this pull request Oct 23, 2024
@kiashok
Merge tag 'v2.0.0-rc.5' into dev/kiashok/update-2.0-rc5
cf261aa 
containerd 2.0.0-rc.5

Welcome to the v2.0.0-rc.5 release of containerd!
*This is a pre-release of containerd*

The first major release of containerd 2.x focuses on the continued stability of
containerd's core feature set with an easy upgrade from containerd 1.x. This
release includes the stabilization of new features added in the last 1.x release
as well as the removal of features which were deprecated in 1.x. The goal is to
support the vast community of containerd users well into the future along with
their ever increasing deployment footprints and variety of use cases.

* Add Update API for sandbox controller ([containerd#9903](containerd#9903))
* Configure otel from env instead of config.toml ([containerd#8970](containerd#8970))
* Enable NRI by default ([containerd#9744](containerd#9744))
* Add PluginInfo to introspection API ([containerd#9442](containerd#9442))
* Remove overlayfs volatile option on temp mounts ([containerd#9555](containerd#9555))
* Expose usage of deprecated features ([containerd#9258](containerd#9258))
* Use Intel ISA-L's igzip if available ([containerd#9200](containerd#9200))
* Introduce top level config migration ([containerd#9223](containerd#9223))
* Add image delete target ([containerd#8989](containerd#8989))
* Remove `LimitNOFILE` from `containerd.service` ([containerd#8924](containerd#8924))
* Add support for image expiration during garbage collection ([containerd#9022](containerd#9022))
* Reduce the contention between ref lock and boltdb lock in content store ([containerd#8792](containerd#8792))
* Remove "containerd.io/restart.logpath" label ([containerd#8264](containerd#8264))
* Remove `aufs` snapshotter ([containerd#8263](containerd#8263))
* Fix deadlock during NRI plugin registration ([containerd/nri#79](containerd/nri#79))
* Fix deadlock when writing to pipe blocks ([containerd/ttrpc#168](containerd/ttrpc#168))

* Generate attestation for artifacts during release ([containerd#10543](containerd#10543))

* Use 'UserSpecifiedImage' from CRI to set the image-name annotation ([containerd#10747](containerd#10747))
* Add support to set loopback to up ([containerd#10238](containerd#10238))
* Add support for multiple subscribers to CRI container events ([containerd#9661](containerd#9661))
* Enable CDI by default ([containerd#9621](containerd#9621))
* Remove non-sandboxed CRI implementation ([containerd#9228](containerd#9228))
* Add support for userns in stateless and stateful pods with idmap mounts (KEP-127, k8s >= 1.27) ([containerd#8287](containerd#8287))
* Use sandboxed CRI by default ([containerd#8994](containerd#8994))
* Implement RuntimeConfig CRI call ([containerd#8722](containerd#8722))
* Add support for user namespaces (KEP-127) ([containerd#8803](containerd#8803))
* Remove CRI v1alpha2 ([containerd#8276](containerd#8276))

* Add api Go module and move all protos under api ([containerd#10151](containerd#10151))
* Move packages based on contributing guide ([containerd#9365](containerd#9365))
* Generalize plugin library ([containerd#9214](containerd#9214))
* Use github.com/containerd/log ([containerd#9086](containerd#9086))

* Support to syncfs after pull by using diff plugin ([containerd#10284](containerd#10284))
* Skip "unknown" in image platform listing ([containerd#10257](containerd#10257))
* Update unpacker to fetch all provided content ([containerd#10202](containerd#10202))
* Enable Transfer service API to support plain HTTP ([containerd#10024](containerd#10024))
* Enable Transfer service to use registry configuration directory ([containerd#9908](containerd#9908))
* Disable the support for Schema 1 images ([containerd#9765](containerd#9765))
* Update Transfer service to add OCI descriptors to Progress structure ([containerd#9630](containerd#9630))
* Update import and export to allow references to missing content  ([containerd#9554](containerd#9554))
* Add option to perform syncfs after pull ([containerd#9401](containerd#9401))
* Add image verifier transfer service plugin system based on a binary directory ([containerd#8493](containerd#8493))

* Implement  RuntimeStatus.features.supplemental_groups_policy from KEP-3619 ([containerd#10410](containerd#10410))
* Add pprof to runc-shim ([containerd#10242](containerd#10242))
* Provide runtime options in plugin info ([containerd#10251](containerd#10251))
* Store bootstrap parameters in sandbox metadata ([containerd#9736](containerd#9736))
* Update apparmor to allow confined runc to kill containers ([containerd#10123](containerd#10123))
* Support vsock connection to task api ([containerd#9738](containerd#9738))
* Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([containerd#9320](containerd#9320))
* Switch runc shim to task service v3 and fix restore ([containerd#9233](containerd#9233))
* Add sandboxer configuration and move sandbox controllers to plugins ([containerd#8268](containerd#8268))
* Add annotations to CreateSandbox request ([containerd#8960](containerd#8960))
* Add SandboxMetrics ([containerd#8680](containerd#8680))
* Publish sandbox events ([containerd#8602](containerd#8602))
* Remove the CriuPath field from runc's options ([containerd#8279](containerd#8279))
* Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([containerd#8262](containerd#8262))

* [medium] RAPL accessible to a container [GHSA-7ww5-4wqc-m92c](GHSA-7ww5-4wqc-m92c)

* Remove `disable_cgroup` from CRI config ([containerd#10594](containerd#10594))
* Disable the support for Schema 1 images ([containerd#9765](containerd#9765))
* Update RuntimeDefault seccomp profile to disallow io_uring related syscalls ([containerd#9320](containerd#9320))
* Move client to subpackage ([containerd#9316](containerd#9316))
* Remove `LimitNOFILE` from `containerd.service` ([containerd#8924](containerd#8924))
* Remove CRI v1alpha2 ([containerd#8276](containerd#8276))
* Remove `io.containerd.runtime.v1.linux` and `io.containerd.runc.v1` ([containerd#8262](containerd#8262))
* Remove "containerd.io/restart.logpath" label ([containerd#8264](containerd#8264))
* Remove `aufs` snapshotter ([containerd#8263](containerd#8263))

* Update warnings for deprecated CRI config fields ([containerd#10509](containerd#10509))
* Add type alias for event Envelope ([containerd#10279](containerd#10279))
* Postpone removal of deprecated CRI config properties ([containerd#9966](containerd#9966))
* Deprecate go-plugin configuration option ([containerd#9238](containerd#9238))
* CNI conf_template in CRI is no longer deprecated ([containerd#8637](containerd#8637))

Please try out the release binaries and report any issues at
https://github.com/containerd/containerd/issues.

* Derek McGowan
* Akihiro Suda
* Maksym Pavlenko
* Wei Fu
* Phil Estes
* Sebastiaan van Stijn
* Samuel Karp
* Stefan Berger
* Kazuyoshi Kato
* Rodrigo Campos
* Danny Canter
* Abel Feng
* Akhil Mohan
* Kirtana Ashok
* Gabriel Adrian Samfira
* Austin Vazquez
* Iceber Gu
* Krisztian Litkey
* Kohei Tokunaga
* Mike Brown
* Jin Dong
* Bjorn Neergaard
* Justin Chadwell
* rongfu.leng
* James Sturtevant
* Davanum Srinivas
* Paul "TBBle" Hampson
* Henry Wang
* Brian Goff
* Enrico Weigelt
* Laura Brehm
* Marat Radchenko
* Paweł Gronowski
* Shingo Omura
* Hsing-Yu (David) Chen
* Ilya Hanov
* Cardy.Tang
* Swagat Bora
* Aditi Sharma
* Amit Barve
* Bryant Biggs
* Evan Lezar
* James Jenkins
* Jordan Liggitt
* Kay Yan
* Markus Lehtonen
* Nashwan Azhari
* Shuaiyi Zhang
* Vinayak Goyal
* helen
* Alexandru Matei
* Anthony Nandaa
* Avi Deitcher
* Charity Kathure
* Cory Snider
* Ed Bartosh
* Etienne Champetier
* Kevin Parsons
* Michael Zappa
* Milas Bowman
* ningmingxiao
* yanggang
* zounengren
* Aditya Ramani
* Adrian Reber
* Amir M. Ghazanfari
* Artem Khramov
* Brad Davidson
* Chen Yiyang
* Christian Muehlhaeuser
* Djordje Lukic
* Edgar Lee
* Eric Lin
* Ethan Lowman
* Jiang Liu
* June Rhodes
* Kern Walster
* Lucas Rattz
* Mahamed Ali
* Maksim An
* Michael Crosby
* Peteris Rudzusiks
* Sam Edwards
* Samruddhi Khandale
* Sascha Grunert
* Steve Griffith
* Tony Fang
* VERNOU Cédric
* Vishal Reddy Gurrala
* hang.jiang
* harshitasao
* jerryzhuang
* lengrongfu
* roman-kiselenko
* zhanluxianshen
* Aaron Lehmann
* Adrien Delorme
* Alex Couture-Beil
* Alex Ellis
* Alex Rodriguez
* Angelos Kolaitis
* Antonio Huete Jimenez
* Arash Haghighat
* Ben Foster
* Bin Tang
* Bin Xin
* BinBin He
* Brennan Kinney
* Changqing Li
* ChengenH
* ChengyuZhu6
* Christian Stewart
* Colin O'Dell
* Craig Ingram
* Daisy Rong
* David Porter
* Derek Nola
* Eng Zer Jun
* Erikson Tung
* Fabiano Fidêncio
* Fahed Dorgaa
* Gary McDonald
* Iain Macdonald
* James Lakin
* Jan Dubois
* Jaroslav Jindrak
* Javier Maestro
* Jian Wang
* Jiongchi Yu
* Julien Balestra
* Kir Kolyshkin
* Kirill A. Korinsky
* Konstantin Khlebnikov
* Mauri de Souza Meneguzzo
* Pan Yibo
* Paul Meyer
* Qasim Sarfraz
* Qiutong Song
* Reinhard Tartler
* Robbie Buxton
* Robert-André Mauchin
* Ruihua Wen
* Sameer
* Shengjing Zhu
* Shiming Zhang
* Shukui Yang
* Talon
* Tariq Ibrahim
* Tianon Gravi
* Tim Hockin
* TinaMor
* Tobias Klauser
* Tomáš Virtus
* Tõnis Tiigi
* Wang Xinwen
* William Chen
* Xinyang Ge
* Yibo Zhuang
* Yury Gargay
* Zechun Chen
* Zhang Tianyang
* Zoe
* baijia
* bo.jiang
* bzsuni
* charles-chenzz
* chschumacher1994
* guangli.bao
* guangwu
* jinda.ljd
* krglosse
* pigletfly
* rokkiter
* wangxiang
* zhangpeng
* zhaojizhuang
* 吴小白
* 张钰
* 沈陵
* 谭九鼎

* **dario.cat/mergo**                                                              v1.0.1 **_new_**
* **github.com/AdaLogics/go-fuzz-headers**                                         1f10f66a31bf -> ced1acdcaa24
* **github.com/AdamKorcz/go-118-fuzz-build**                                       5330a85ea652 -> 8075edf89bb0
* **github.com/Microsoft/go-winio**                                                v0.6.0 -> v0.6.2
* **github.com/Microsoft/hcsshim**                                                 v0.10.0-rc.7 -> v0.12.6
* **github.com/cenkalti/backoff/v4**                                               v4.2.0 -> v4.3.0
* **github.com/cespare/xxhash/v2**                                                 v2.2.0 -> v2.3.0
* **github.com/checkpoint-restore/checkpointctl**                                  v1.2.1 **_new_**
* **github.com/checkpoint-restore/go-criu/v7**                                     v7.2.0 **_new_**
* **github.com/cilium/ebpf**                                                       v0.9.1 -> v0.11.0
* **github.com/containerd/cgroups/v3**                                             v3.0.1 -> v3.0.3
* **github.com/containerd/console**                                                v1.0.3 -> v1.0.4
* **github.com/containerd/containerd/api**                                         v1.8.0-rc.3 **_new_**
* **github.com/containerd/continuity**                                             v0.3.0 -> v0.4.3
* **github.com/containerd/errdefs**                                                v0.1.0 **_new_**
* **github.com/containerd/go-cni**                                                 v1.1.9 -> v1.1.10
* **github.com/containerd/go-runc**                                                v1.0.0 -> v1.1.0
* **github.com/containerd/imgcrypt**                                               v1.1.7 -> v1.2.0-rc1
* **github.com/containerd/log**                                                    v0.1.0 **_new_**
* **github.com/containerd/nri**                                                    v0.3.0 -> v0.6.1
* **github.com/containerd/otelttrpc**                                              ea5083fda723 **_new_**
* **github.com/containerd/platforms**                                              v0.2.1 **_new_**
* **github.com/containerd/plugin**                                                 v0.1.0 **_new_**
* **github.com/containerd/ttrpc**                                                  v1.2.1 -> v1.2.5
* **github.com/containerd/typeurl/v2**                                             v2.1.0 -> v2.2.0
* **github.com/containernetworking/cni**                                           v1.1.2 -> v1.2.3
* **github.com/containernetworking/plugins**                                       v1.2.0 -> v1.5.1
* **github.com/containers/ocicrypt**                                               v1.1.6 -> v1.2.0
* **github.com/cpuguy83/go-md2man/v2**                                             v2.0.2 -> v2.0.4
* **github.com/davecgh/go-spew**                                                   v1.1.1 -> d8f796af33cc
* **github.com/distribution/reference**                                            v0.6.0 **_new_**
* **github.com/emicklei/go-restful/v3**                                            v3.10.1 -> v3.11.0
* **github.com/felixge/httpsnoop**                                                 v1.0.4 **_new_**
* **github.com/fsnotify/fsnotify**                                                 v1.6.0 -> v1.7.0
* **github.com/fxamacker/cbor/v2**                                                 v2.7.0 **_new_**
* **github.com/go-jose/go-jose/v4**                                                v4.0.2 **_new_**
* **github.com/go-logr/logr**                                                      v1.2.3 -> v1.4.2
* **github.com/golang/protobuf**                                                   v1.5.2 -> v1.5.4
* **github.com/google/go-cmp**                                                     v0.5.9 -> v0.6.0
* **github.com/google/uuid**                                                       v1.3.0 -> v1.6.0
* **github.com/gorilla/websocket**                                                 v1.5.0 **_new_**
* **github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus**            v1.0.1 **_new_**
* **github.com/grpc-ecosystem/go-grpc-middleware/v2**                              v2.1.0 **_new_**
* **github.com/grpc-ecosystem/grpc-gateway/v2**                                    v2.7.0 -> v2.22.0
* **github.com/intel/goresctrl**                                                   v0.3.0 -> v0.7.0
* **github.com/klauspost/compress**                                                v1.16.0 -> v1.17.10
* **github.com/mdlayher/socket**                                                   v0.4.1 **_new_**
* **github.com/mdlayher/vsock**                                                    v1.2.1 **_new_**
* **github.com/moby/spdystream**                                                   v0.2.0 -> v0.4.0
* **github.com/moby/sys/mountinfo**                                                v0.6.2 -> v0.7.2
* **github.com/moby/sys/sequential**                                               v0.5.0 -> v0.6.0
* **github.com/moby/sys/signal**                                                   v0.7.0 -> v0.7.1
* **github.com/moby/sys/symlink**                                                  v0.2.0 -> v0.3.0
* **github.com/moby/sys/user**                                                     v0.3.0 **_new_**
* **github.com/moby/sys/userns**                                                   v0.1.0 **_new_**
* **github.com/munnerz/goautoneg**                                                 a7dc8b61c822 **_new_**
* **github.com/mxk/go-flowrate**                                                   cca7078d478f **_new_**
* **github.com/opencontainers/image-spec**                                         3a7f492d3f1b -> v1.1.0
* **github.com/opencontainers/runtime-spec**                                       v1.1.0-rc.1 -> v1.2.0
* **github.com/opencontainers/runtime-tools**                                      946c877fa809 -> 2e043c6bd626
* **github.com/pelletier/go-toml/v2**                                              v2.2.3 **_new_**
* **github.com/pmezard/go-difflib**                                                v1.0.0 -> 5d4384ee4fb2
* **github.com/prometheus/client_golang**                                          v1.14.0 -> v1.20.4
* **github.com/prometheus/client_model**                                           v0.3.0 -> v0.6.1
* **github.com/prometheus/common**                                                 v0.37.0 -> v0.55.0
* **github.com/prometheus/procfs**                                                 v0.8.0 -> v0.15.1
* **github.com/sirupsen/logrus**                                                   v1.9.0 -> v1.9.3
* **github.com/stretchr/testify**                                                  v1.8.2 -> v1.9.0
* **github.com/urfave/cli/v2**                                                     v2.27.4 **_new_**
* **github.com/vishvananda/netlink**                                               v1.2.1-beta.2 -> v1.3.0
* **github.com/vishvananda/netns**                                                 2eb08e3e575f -> v0.0.4
* **github.com/x448/float16**                                                      v0.8.4 **_new_**
* **github.com/xrash/smetrics**                                                    686a1a2994c1 **_new_**
* **go.etcd.io/bbolt**                                                             v1.3.7 -> v1.3.11
* **go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc**  v0.40.0 -> v0.55.0
* **go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp**                v0.55.0 **_new_**
* **go.opentelemetry.io/otel**                                                     v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace**                            v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc**              v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp**              v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/metric**                                              v0.37.0 -> v1.30.0
* **go.opentelemetry.io/otel/sdk**                                                 v1.14.0 -> v1.30.0
* **go.opentelemetry.io/otel/trace**                                               v1.14.0 -> v1.30.0
* **go.opentelemetry.io/proto/otlp**                                               v0.19.0 -> v1.3.1
* **golang.org/x/crypto**                                                          v0.1.0 -> v0.27.0
* **golang.org/x/exp**                                                             aacd6d4b4611 **_new_**
* **golang.org/x/mod**                                                             v0.7.0 -> v0.21.0
* **golang.org/x/net**                                                             v0.7.0 -> v0.29.0
* **golang.org/x/oauth2**                                                          v0.4.0 -> v0.22.0
* **golang.org/x/sync**                                                            v0.1.0 -> v0.8.0
* **golang.org/x/sys**                                                             v0.6.0 -> v0.25.0
* **golang.org/x/term**                                                            v0.5.0 -> v0.24.0
* **golang.org/x/text**                                                            v0.7.0 -> v0.18.0
* **golang.org/x/time**                                                            90d013bbcef8 -> v0.3.0
* **google.golang.org/genproto/googleapis/api**                                    8af14fe29dc1 **_new_**
* **google.golang.org/genproto/googleapis/rpc**                                    8af14fe29dc1 **_new_**
* **google.golang.org/grpc**                                                       v1.53.0 -> v1.67.0
* **google.golang.org/protobuf**                                                   v1.28.1 -> v1.34.2
* **k8s.io/api**                                                                   v0.26.2 -> v0.31.1
* **k8s.io/apimachinery**                                                          v0.26.2 -> v0.31.1
* **k8s.io/apiserver**                                                             v0.26.2 -> v0.31.1
* **k8s.io/client-go**                                                             v0.26.2 -> v0.31.1
* **k8s.io/component-base**                                                        v0.26.2 -> v0.31.1
* **k8s.io/cri-api**                                                               v0.26.2 -> v0.32.0-alpha.0
* **k8s.io/klog/v2**                                                               v2.90.1 -> v2.130.1
* **k8s.io/kubelet**                                                               v0.31.1 **_new_**
* **k8s.io/utils**                                                                 a5ecb0141aa5 -> 18e509b52bc8
* **sigs.k8s.io/json**                                                             f223a00ba0e2 -> bc3834ca7abd
* **sigs.k8s.io/structured-merge-diff/v4**                                         v4.2.3 -> v4.4.1
* **sigs.k8s.io/yaml**                                                             v1.3.0 -> v1.4.0
* **tags.cncf.io/container-device-interface**                                      v0.8.0 **_new_**
* **tags.cncf.io/container-device-interface/specs-go**                             v0.8.0 **_new_**

Previous release can be found at [v1.7.0](https://github.com/containerd/containerd/releases/tag/v1.7.0)
* `containerd-<VERSION>-<OS>-<ARCH>.tar.gz`:         ✅Recommended. Dynamically linked with glibc 2.31 (Ubuntu 20.04).
* `containerd-static-<VERSION>-<OS>-<ARCH>.tar.gz`:  Statically linked. Expected to be used on non-glibc Linux distributions. Not position-independent.

In addition to containerd, typically you will have to install [runc](https://github.com/opencontainers/runc/releases)
and [CNI plugins](https://github.com/containernetworking/plugins/releases) from their official sites too.

See also the [Getting Started](https://github.com/containerd/containerd/blob/main/docs/getting-started.md) documentation.
@dmcgowan dmcgowan mentioned this pull request Oct 23, 2024
Merged
@dmcgowan dmcgowan mentioned this pull request Nov 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@estesp estesp estesp approved these changes

@samuelkarp samuelkarp samuelkarp approved these changes

Assignees
No one assigned
Labels
impact/changelog
Projects
Pull Request Review
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

NRI plugin registration can trigger a deadlock
5 participants
@bobbypage @samuelkarp @klihub @estesp @dmcgowan