Paguro v0.1.3

Bottlerocket and EKS Auto Mode with nothing to set up (beta), SELinux with
a label per pod, and a new license: Apache-2.0.

Added:

• SELinux on nodes whose container runtime labels every pod with
  categories of its own (containerd with enable_selinux = true and
  container-selinux, as on RHEL and Bottlerocket). Before, the first
  pre-dump was denied – CRIU, built without libselinux, could not label
  its parasite's socket – and the migration rolled back. Now CRIU is built
  with libselinux; its work directories and logs carry the container's
  file label; restored processes and mounts get the replacement's labels
  (--lsm-profile, --lsm-mount-context); files restored into volumes
  take the volume's label on the target node instead of the source's.
  Tested on EKS 1.37, Amazon Linux 2023, SELinux enforcing: freeze
  381/408 ms, the replacement with categories of its own, no denial.
  Enforcing without labelling (the AMI's containerd configuration) and
  permissive (its default) worked before and still do.
• Bottlerocket (beta), with nothing to set up: the same helm install. The
  controller labels Bottlerocket nodes paguro.dev/os=bottlerocket, and
  the chart runs the agent there as paguro-agent-bottlerocket with the
  SELinux type super_t (Bottlerocket starts privileged containers as
  control_t, which may not prepare the node). The node installer works
  with a read-only root (paguro-runc and its tools in /opt/paguro),
  without a host shell, with containerd's config.d and with
  Bottlerocket's second containerd; the bundle brings nsenter, ip,
  ss, cat, tee, rm and tar. Checkpoints run in the runtime's SELinux
  context (runtime_t, through a transient systemd unit), where CRIU's
  parasite may hand over the pod's memory. Three CRIU patches, no-ops
  elsewhere: the pod's process clears its own soft-dirty bits (no other
  domain may write its clear_refs); tmpfs contents are archived as plain
  tar (GNU tar starts gzip through /bin/sh, which Bottlerocket's shell
  refuses; Paguro compresses its transfers anyway, and gzip ran inside the
  freeze); restored memory becomes executable only in the pod's domain
  (runtime_t may never map memory writable and executable). Tested on
  EKS 1.37 with Bottlerocket 1.67.0: four migrations of a pod with state
  in memory, emptyDir and root filesystem (freeze 366–459 ms) and two of a
  JVM with JIT-compiled code, each with SELinux categories of its own on
  the target. With clients in Phantom mode (2026-10-27, EKS cluster with
  two Bottlerocket and two AL2023 nodes): a UDP game server and a TCP
  echo server, each moved back and forth; players through the ClusterIP
  and a NodePort kept their sessions (36–48 of 3000 inputs lost, within
  the freeze), no new TCP connection through the ClusterIP failed, the
  long-lived connection survived – as on AL2023. A rebooted node gets its
  handler again from the agent (one containerd restart) and takes and
  gives migrations; an EBS volume moves with the pod; helm uninstall
  and the documented node cleanup leave the node as it was.
• EKS Auto Mode (beta): its nodes are Bottlerocket, and Paguro runs there
  the same way, nothing to set up. Three migrations, freeze 315–406 ms.
• In clusters with Bottlerocket and other nodes, pods move only between
  nodes of the same kind: a checkpoint does not restore across them
  (tried on EKS: CRIU fails on SELinux contexts one side does not know and
  on the containers' mounts, and the replacement cold-starts). Placement
  never picks such a target; an explicit target is refused.

Changed:

• License: Apache-2.0 instead of AGPL-3.0-only – Paguro may now be built
  into other products, proprietary ones included, as long as the license
  and the notices (NOTICE) go with it. Releases up to and including 0.1.2
  stay under AGPL-3.0-only. The eBPF program of Phantom mode keeps
  GPL-2.0-only OR BSD-2-Clause, the patches to CRIU GPL-2.0-only.
• The chart's controller may patch nodes – only the label paguro.dev/os,
  held to that by an admission policy (agent.restrictionPolicy.enabled;
  checked on Kubernetes 1.30–1.37).
• tmpfs contents in a checkpoint are plain tar instead of gzip'ed tar:
  gzip ran inside the freeze, and Paguro compresses what it transfers
  anyway.

Fixed:

• Security: a user who may create pods could take over a node
  (releases 0.1.0–0.1.2). paguro-runc took the pod annotation
  paguro.dev/restore-id as a directory name: with ../ in it and
  runtimeClassName: paguro, a pod made paguro-runc – root on the node –
  restore a checkpoint from a directory the pod controlled (its own
  emptyDir), or claim another namespace's waiting checkpoint. Now the
  annotation must be a migration UID, a waiting checkpoint is restored
  only into a pod of the migration's namespace (recorded by the target
  agent), the webhook admits new pods carrying the annotation only from
  Paguro's controller, and an admission policy keeps anyone from adding
  or changing it on an existing pod. Found in an internal review; tried
  on EKS: the attempts are refused by the webhook, the policy and – with
  both bypassed – by paguro-runc on the node.
• Security: the node installer could run a pod's program as root
  (releases 0.1.0–0.1.2) on
  hosts without a containerd.service unit (k3s, RKE2). It took the
  first process named containerd for the node's containerd and ran
  programs from that process's PATH and binary; a pod can name a process
  so. Only a system service (its cgroup a *.service outside kubepods)
  that sees the host's root qualifies now. Not affected: hosts where
  containerd runs as containerd.service (kubeadm, EKS AL2023,
  Bottlerocket).
• Known vulnerabilities: Go 1.26.10 for Paguro's binaries (GO-2026-6809
  in crypto/x509); crictl in the node installer is rebuilt from its
  v1.37.0 source with Go 1.26.10 and golang.org/x/net v0.60.0 – the
  release binary carries net/http and x/net vulnerabilities its code
  reaches (govulncheck).

Known limitations:

• An RWO EBS volume moves inside the freeze: measured 17–24 s on AL2023
  and 25–45 s on Bottlerocket (detach and attach). io2 Multi-Attach
  takes the attach out of the freeze.
• Bottlerocket: every migration logs two expected SELinux denials –
  CRIU's first attempt to write the pod's clear_refs and to map
  executable memory as runtime_t, before it takes the way the policy
  allows. Security tooling may report them.
• Pods do not move between Bottlerocket and other nodes (above).

Security advisories

• GHSA-jj9m-449r-pxjj (critical): node takeover by a user who can create pods – paguro.dev/restore-id taken as a path. Affects 0.1.0–0.1.2.
• GHSA-vj5j-q9v9-4c9r (high): node installer could run a pod's program as root on hosts without containerd.service (k3s, RKE2). Affects 0.1.0–0.1.2.

Upgrade: helm upgrade paguro oci://ghcr.io/dpicillo/charts/paguro --version 0.1.3 -n paguro-system

Paguro v0.1.2

Fixed:

• Phantom mode: a connection whose pod comes back to a node and an address
  it held there before (the AWS VPC CNI hands a freed address out again)
  no longer hangs. The node still had the connection's conntrack entries
  from the pod's earlier stay, with old sequence numbers; window tracking
  rated every packet INVALID and kube-proxy dropped it, in both directions.
  Connections with little traffic stayed inside the old window and were
  not affected. The target node now deletes such entries – exactly the
  migrated flow's tuple, never one with NAT – before it translates.

Added:

• The README and the package page show recordings of a Minecraft Java
  Edition (TCP) and a Red Eclipse (UDP) server live-migrating on Amazon
  EKS: about 2 s per migration, 0.3–0.4 s frozen, players stay connected.

Paguro v0.1.1

Documentation only: the chart's README and its Artifact Hub page no longer
show the recording of a migration with a 25 GiB block volume, whose freeze
of about 12 s (the volume's detach and attach) is not typical. Install
commands point to 0.1.1. No code changes.

Paguro v0.1.0

First public release. Paguro live-migrates running Kubernetes pods between
nodes with CRIU: memory, open files, root filesystem changes, emptyDirs
and volumes move with the pod, and its TCP connections and UDP flows
continue – with the pod IP kept where the CNI can move it (Cilium, Calico)
and with Phantom mode everywhere else. The API is paguro.dev/v1alpha1
(alpha).

helm install paguro oci://ghcr.io/dpicillo/charts/paguro --version 0.1.0 \
  -n paguro-system --create-namespace

Installation: docs/INSTALL.md. The story behind Paguro:
https://www.picillo.de/blog/kubernetes-pod-live-migration/

Migration

• Migration resource and the kubectl plugin (kubectl paguro migrate,
  drain, list, describe, nodes, version); workloads opt in with
  the pod label paguro.dev/migratable: "true". The network mode is chosen
  per migration (spec.network) or per workload (annotation
  paguro.dev/network): Auto, Preserve, Phantom or Generic; an
  unknown value is refused, never read as Auto.
• Pre-copy in rounds while the pod runs, with auto-converge (CPU
  throttling when the rounds would not reach the freeze budget); the final
  dump happens in a freeze that clients see as a short stall. A round that
  cannot freeze a process in uninterruptible I/O (NFS) starts over, twice
  at most, before the migration rolls back.
• A commit point: every failure before it rolls back and the source simply
  continues; after it the replacement is the workload (the restore is
  retried, a cold start is the last resort).
• Replacements for Deployments (created during pre-copy), StatefulSets,
  bare pods and other owners (under the same name), and Agones
  GameServers.
• Lazy restore for pods from 64 MiB on; CRIU patched for fast page faults
  on large, fragmented memory.
• A CPU baseline for migratable pods, so that they can move between CPU
  generations.
• Chained migrations: a pod moves again and again, also back to a node and
  an address it had before.

Volumes and files

• RWO volumes detached and attached around the freeze, attached during
  pre-copy for multi-attach volume types (opt-in); RWX volumes written back
  before the final dump, file locks on shared filesystems handed over,
  files deleted while open on NFS migrated.
• emptyDirs copied during pre-copy; the freeze carries only what changed.
  The pod's own /dev/shm (the sandbox's tmpfs, not a volume) moves too.
• Container images pinned to the digest the source ran.

Networking

• Pod IP kept on Cilium (multi-pool IPAM: a sticky /32 per pod, handed
  over early) and Calico (calico-ipam, with a route guard and a NAT
  guard for NodePort and LoadBalancer clients); every connection survives,
  also to peers outside the cluster.
• Phantom mode where the IP cannot move: the pod gets a new IP, and
  eBPF programs on the nodes involved translate the old address of each
  migrated in-cluster connection (TCP and UDP). Works with any CNI –
  Flannel, Antrea, the AWS VPC CNI and others. Connections it cannot keep
  are aborted right after the restore, so the application reconnects at
  once. A peer node routes the old address only for the migrated
  connections, so an address the CNI hands to another pod reaches its new
  owner – at a cost per packet that does not grow with their number: the
  forwarded ones (pods behind a ClusterIP, NodePort clients) are selected by
  one bit of the packet mark (phantom.steeringMark, default 0x2000) and
  a single policy rule, not by a rule each.
• UDP servers that know their players by address keep them when the pod
  gets a new IP, also players behind a NodePort or LoadBalancer whose
  source port the node masquerades with --random-fully (measured on
  Amazon EKS with real Xonotic and Red Eclipse clients behind an NLB: no
  session lost in 37 migrations).
• Services keep the migrating pod as an endpoint until its replacement
  serves – with a kept and with a new IP – so new connections during a
  migration are delayed, not refused; the replacement's readiness probe
  starts without its initial delay. In Phantom mode a connection opened
  through a Service during the freeze is handed back to the Service after
  the restore instead of hanging until the client's timeout.
• Commit gate (Dynamic Resource Allocation, Kubernetes ≥ 1.34): with a kept
  IP the replacement's sandbox is created the moment the migration
  commits.

Drains, autoscalers and spot

• Evictions of migratable pods become migrations: kubectl drain,
  Karpenter (drift, consolidation, expiry, spot interruptions), the
  Cluster Autoscaler and managed node group upgrades move such pods
  instead of restarting them. A migration that fails lets the next
  eviction through.
• At most three migrations leave a node at a time
  (controller.migrationsPerNode); an eviction's migration waits up to two
  minutes for a node to move to (controller.evictionTargetWait) – the
  one Karpenter launches.
• On EC2 a spot interruption's deadline (paguro.dev/terminates-at)
  decides whether a migration still fits; otherwise the eviction goes
  through at once.
• Startup taint paguro.dev/agent-not-ready for Karpenter NodePools: new
  nodes count as initialized once Paguro's agent runs there.

Security

• Mutual TLS 1.3 between agents with a certificate per node, signed by the
  controller only for the requester's own node; a target takes a
  migration's data only from its source node.
• Admission policies hold each agent's service account to its own node
  (Kubernetes ≥ 1.30).
• Checkpoints are removed after every migration, and by a janitor.
• Images and chart signed with cosign (keyless), SPDX SBOM attestations,
  govulncheck and grype on every release. Threat model: SECURITY.md.

Operations

• One Helm chart; its node installer prepares containerd, CRIU and
  nftables on every node and undoes exactly its changes on uninstall.
• Upgrades during running migrations: a stopping agent first finishes the
  migrations on its node, and only agents of the same release migrate
  between each other.
• Prometheus metrics, alert rules, PodMonitors and a Grafana dashboard
  (docs/OPERATIONS.md).
• The agent requests 200m CPU: its weight on a busy node, where CRIU and
  the compression of the pre-copy run in its cgroup
  (agent.resources.requests.cpu; raise it for busy nodes and large pods).

Tested with

• Kubernetes 1.28–1.37: install, webhook, CRD validation, preflight,
  admission policies, commit gate and agent certificates on kind for every
  minor version; migrations on 1.37.
• kubeadm clusters with Cilium, Calico, Flannel and Antrea; containerd
  2.0, 2.1 and 2.4; Ubuntu 22.04 and 24.04, Debian 12, Amazon Linux 2023.
• Amazon EKS 1.37 with Amazon Linux 2023 nodes, the AWS VPC CNI and
  Karpenter 1.14 (drift, NodeClaim deletion, spot interruption warnings).
• Agones 1.61.

Freezes measured: 0.55–0.73 s with the IP kept on Cilium, 1.23–1.58 s on
Calico, 0.3–0.9 s in Phantom mode (small pods, kubeadm lab); on EKS,
Minecraft Java and Bedrock servers with eight players each 0.44–0.66 s,
no player disconnected. More in the README and docs/CNI.md.

Known limitations

• Phantom mode keeps in-cluster connections only: peers outside the
  cluster – on the AWS VPC CNI also NLB/ALB IP targets and other EC2
  instances that reach the pod IP – are closed cleanly.
• Phantom mode: when the restore fails and the replacement cold-starts,
  the in-cluster peers' connections are aborted at once; clients outside
  the cluster (through a NodePort or LoadBalancer) wait for their own
  timeout.
• Phantom mode: a connection opened through a Service during the freeze
  completes about 3 s after the attempt (with a later SYN
  retransmission), and one opened to the old pod IP itself is not
  translated; UDP clients that write to the pod IP directly (not through a
  Service) lose the session; workloads that hand out their own pod IP
  (Kafka advertised.listeners, peer URLs) need a CNI that keeps the IP.
• Players connected through an Agones host port lose the connection (the
  node's address changes).
• Containers with stdin or a TTY are refused (every GitLab Runner job pod);
  processes of kubectl exec sessions do not survive a migration.
• A unix socket connection that is not accepted yet and already holds data
  fails CRIU's final dump; the migration rolls back (GitLab under load).
• A lazy restore can fail in CRIU's restorer with vdso: Invalid ELF magic (GitLab's runit processes on EKS after 5–10 pre-copy rounds; not
  reproduced with the same processes in small pods). The restore wrapper
  then tries once more without lazy pages before it falls back to a cold
  start; the cause is not found yet.
• Memory written nearly as fast as the network copies it gives long
  freezes (8 GiB rewritten at 50 MB/s over 1 GbE: 21–33 s).
• An RWO block volume adds its detach to the freeze (6–9 s on Cinder); EBS
  volumes move only within their availability zone.
• A spot interruption leaves two minutes; a migration that does not fit
  lets the eviction through. In one of five spot runs on EKS the players
  lost the connection about 20 s after a successful restore (not
  reproduced since).
• Not supported: arm64 (not yet), Bottlerocket, EKS Auto Mode, Fargate,
  Windows, CRI-O and Docker. Not run yet: GKE, AKS, service meshes,
  Calico's eBPF dataplane, clusters of more than five nodes.

Upgrading

First release – nothing to upgrade from. Before 1.0 a minor version may
change the API and the chart's values; each release's notes say what to
do.

Paguro v0.1.0-rc.6

First public release. Paguro live-migrates running Kubernetes pods between
nodes with CRIU: memory, open files, root filesystem changes, emptyDirs
and volumes move with the pod, and its TCP connections and UDP flows
continue – with the pod IP kept where the CNI can move it (Cilium, Calico)
and with Phantom mode everywhere else. The API is paguro.dev/v1alpha1
(alpha).

helm install paguro oci://ghcr.io/dpicillo/charts/paguro --version 0.1.0 \
  -n paguro-system --create-namespace

Installation: docs/INSTALL.md. The story behind Paguro:
https://www.picillo.de/blog/kubernetes-pod-live-migration/

Migration

• Migration resource and the kubectl plugin (kubectl paguro migrate,
  drain, list, describe, nodes, version); workloads opt in with
  the pod label paguro.dev/migratable: "true". The network mode is chosen
  per migration (spec.network) or per workload (annotation
  paguro.dev/network): Auto, Preserve, Phantom or Generic; an
  unknown value is refused, never read as Auto.
• Pre-copy in rounds while the pod runs, with auto-converge (CPU
  throttling when the rounds would not reach the freeze budget); the final
  dump happens in a freeze that clients see as a short stall. A round that
  cannot freeze a process in uninterruptible I/O (NFS) starts over, twice
  at most, before the migration rolls back.
• A commit point: every failure before it rolls back and the source simply
  continues; after it the replacement is the workload (the restore is
  retried, a cold start is the last resort).
• Replacements for Deployments (created during pre-copy), StatefulSets,
  bare pods and other owners (under the same name), and Agones
  GameServers.
• Lazy restore for pods from 64 MiB on; CRIU patched for fast page faults
  on large, fragmented memory.
• A CPU baseline for migratable pods, so that they can move between CPU
  generations.
• Chained migrations: a pod moves again and again, also back to a node and
  an address it had before.

Volumes and files

• RWO volumes detached and attached around the freeze, attached during
  pre-copy for multi-attach volume types (opt-in); RWX volumes written back
  before the final dump, file locks on shared filesystems handed over,
  files deleted while open on NFS migrated.
• emptyDirs copied during pre-copy; the freeze carries only what changed.
  The pod's own /dev/shm (the sandbox's tmpfs, not a volume) moves too.
• Container images pinned to the digest the source ran.

Networking

• Pod IP kept on Cilium (multi-pool IPAM: a sticky /32 per pod, handed
  over early) and Calico (calico-ipam, with a route guard and a NAT
  guard for NodePort and LoadBalancer clients); every connection survives,
  also to peers outside the cluster.
• Phantom mode where the IP cannot move: the pod gets a new IP, and
  eBPF programs on the nodes involved translate the old address of each
  migrated in-cluster connection (TCP and UDP). Works with any CNI –
  Flannel, Antrea, the AWS VPC CNI and others. Connections it cannot keep
  are aborted right after the restore, so the application reconnects at
  once. A peer node routes the old address only for the migrated
  connections, so an address the CNI hands to another pod reaches its new
  owner – at a cost per packet that does not grow with their number: the
  forwarded ones (pods behind a ClusterIP, NodePort clients) are selected by
  one bit of the packet mark (phantom.steeringMark, default 0x2000) and
  a single policy rule, not by a rule each.
• UDP servers that know their players by address keep them when the pod
  gets a new IP, also players behind a NodePort or LoadBalancer whose
  source port the node masquerades with --random-fully (measured on
  Amazon EKS with real Xonotic and Red Eclipse clients behind an NLB: no
  session lost in 37 migrations).
• Services keep the migrating pod as an endpoint until its replacement
  serves – with a kept and with a new IP – so new connections during a
  migration are delayed, not refused; the replacement's readiness probe
  starts without its initial delay. In Phantom mode a connection opened
  through a Service during the freeze is handed back to the Service after
  the restore instead of hanging until the client's timeout.
• Commit gate (Dynamic Resource Allocation, Kubernetes ≥ 1.34): with a kept
  IP the replacement's sandbox is created the moment the migration
  commits.

Drains, autoscalers and spot

• Evictions of migratable pods become migrations: kubectl drain,
  Karpenter (drift, consolidation, expiry, spot interruptions), the
  Cluster Autoscaler and managed node group upgrades move such pods
  instead of restarting them. A migration that fails lets the next
  eviction through.
• At most three migrations leave a node at a time
  (controller.migrationsPerNode); an eviction's migration waits up to two
  minutes for a node to move to (controller.evictionTargetWait) – the
  one Karpenter launches.
• On EC2 a spot interruption's deadline (paguro.dev/terminates-at)
  decides whether a migration still fits; otherwise the eviction goes
  through at once.
• Startup taint paguro.dev/agent-not-ready for Karpenter NodePools: new
  nodes count as initialized once Paguro's agent runs there.

Security

• Mutual TLS 1.3 between agents with a certificate per node, signed by the
  controller only for the requester's own node; a target takes a
  migration's data only from its source node.
• Admission policies hold each agent's service account to its own node
  (Kubernetes ≥ 1.30).
• Checkpoints are removed after every migration, and by a janitor.
• Images and chart signed with cosign (keyless), SPDX SBOM attestations,
  govulncheck and grype on every release. Threat model: SECURITY.md.

Operations

• One Helm chart; its node installer prepares containerd, CRIU and
  nftables on every node and undoes exactly its changes on uninstall.
• Upgrades during running migrations: a stopping agent first finishes the
  migrations on its node, and only agents of the same release migrate
  between each other.
• Prometheus metrics, alert rules, PodMonitors and a Grafana dashboard
  (docs/OPERATIONS.md).
• The agent requests 200m CPU: its weight on a busy node, where CRIU and
  the compression of the pre-copy run in its cgroup
  (agent.resources.requests.cpu; raise it for busy nodes and large pods).

Tested with

• Kubernetes 1.28–1.37: install, webhook, CRD validation, preflight,
  admission policies, commit gate and agent certificates on kind for every
  minor version; migrations on 1.37.
• kubeadm clusters with Cilium, Calico, Flannel and Antrea; containerd
  2.0, 2.1 and 2.4; Ubuntu 22.04 and 24.04, Debian 12, Amazon Linux 2023.
• Amazon EKS 1.37 with Amazon Linux 2023 nodes, the AWS VPC CNI and
  Karpenter 1.14 (drift, NodeClaim deletion, spot interruption warnings).
• Agones 1.61.

Freezes measured: 0.55–0.73 s with the IP kept on Cilium, 1.23–1.58 s on
Calico, 0.3–0.9 s in Phantom mode (small pods, kubeadm lab); on EKS,
Minecraft Java and Bedrock servers with eight players each 0.44–0.66 s,
no player disconnected. More in the README and docs/CNI.md.

Known limitations

• Phantom mode keeps in-cluster connections only: peers outside the
  cluster – on the AWS VPC CNI also NLB/ALB IP targets and other EC2
  instances that reach the pod IP – are closed cleanly.
• Phantom mode: when the restore fails and the replacement cold-starts,
  the in-cluster peers' connections are aborted at once; clients outside
  the cluster (through a NodePort or LoadBalancer) wait for their own
  timeout.
• Phantom mode: a connection opened through a Service during the freeze
  completes about 3 s after the attempt (with a later SYN
  retransmission), and one opened to the old pod IP itself is not
  translated; UDP clients that write to the pod IP directly (not through a
  Service) lose the session; workloads that hand out their own pod IP
  (Kafka advertised.listeners, peer URLs) need a CNI that keeps the IP.
• Players connected through an Agones host port lose the connection (the
  node's address changes).
• Containers with stdin or a TTY are refused (every GitLab Runner job pod);
  processes of kubectl exec sessions do not survive a migration.
• A unix socket connection that is not accepted yet and already holds data
  fails CRIU's final dump; the migration rolls back (GitLab under load).
• A lazy restore can fail in CRIU's restorer with vdso: Invalid ELF magic (GitLab's runit processes on EKS after 5–10 pre-copy rounds; not
  reproduced with the same processes in small pods). The restore wrapper
  then tries once more without lazy pages before it falls back to a cold
  start; the cause is not found yet.
• Memory written nearly as fast as the network copies it gives long
  freezes (8 GiB rewritten at 50 MB/s over 1 GbE: 21–33 s).
• An RWO block volume adds its detach to the freeze (6–9 s on Cinder); EBS
  volumes move only within their availability zone.
• A spot interruption leaves two minutes; a migration that does not fit
  lets the eviction through. In one of five spot runs on EKS the players
  lost the connection about 20 s after a successful restore (not
  reproduced since).
• Not supported: arm64 (not yet), Bottlerocket, EKS Auto Mode, Fargate,
  Windows, CRI-O and Docker. Not run yet: GKE, AKS, service meshes,
  Calico's eBPF dataplane, clusters of more than five nodes.

Upgrading

First release – nothing to upgrade from. Before 1.0 a minor version may
change the API and the chart's values; each release's notes say what to
do.