Containerd and Kubernetes

Mirrored from docs/containerd-kubernetes.md in the repository, 2026-08-05. The repo copy is authoritative for anything CI-cited (exact commands, workflow files, versions) — this page exists so the wiki is self-contained, but re-sync it from the source if the two drift.

containerd and Kubernetes MicroVM runtime

This integration keeps both execution paths available: ordinary workloads keep their existing runtime handler, while workloads selecting platform-factory execute the OCI bundle in a native KVM MicroVM. It does not launch a Docker or Podman container around the VM.

Why a custom shim, not io.containerd.runc.v2 + BinaryName

An earlier version of this integration pointed containerd's stock io.containerd.runc.v2 shim at platform-factory-runtime via its BinaryName option, reusing containerd's default "podsandbox" model: the pause container that anchors each Kubernetes Pod's shared network/IPC namespaces is created the same way any other container is, from an OCI spec containerd's own CRI plugin generates.

That model unconditionally assigns the pause container's spec a non-empty Linux capability set - even with securityContext.capabilities.drop_capabilities: ["ALL"] set on the Pod, and even with containerd's base_runtime_spec override loaded (confirmed in containerd's own debug log: the base spec's process.capabilities is present, but is unconditionally overwritten by containerd's sandbox-generation code afterward). platform-factory-runtime deliberately refuses any OCI spec carrying a non-empty capability set - a MicroVM's guest kernel does not admit host Linux capabilities into its own boot process at all, so honoring them would be lying about what protection they buy. Relaxing that refusal to accommodate containerd's sandbox model was rejected: the runtime stays strict, and the integration works around containerd instead.

platform-factory-shim (installed as containerd-shim-platform-factory-v1, matching containerd's naming convention for the io.containerd.platform-factory.v1 runtime_type below) is a full containerd runtime v2 shim implementing both TTRPC services containerd's sandboxer = "shim" model requires:

• a Sandbox service that never presents any OCI spec for containerd to attach capabilities to in the first place - it does no more than track a pod's bundle path, network namespace, and lifecycle timestamps in memory; nothing about it needs a capability policy to reject.
• a Task service, one per container in the pod, that shells out to the same platform-factory-runtime CLI Podman already drives in production (create/start/state/kill/delete), so MicroVM lifecycle logic is never duplicated.

Every container is still its own independently isolated MicroVM; the shim adds no shared kernel, VM, or process tree across containers in a pod.

Installation

On a Linux amd64 KVM node:

sudo scripts/microvm/install-containerd-runtime.sh

This builds and installs three binaries: platform-factory-runtime (the OCI CLI facade, from the main module), and containerd-shim-platform-factory-v1 and platform-factory-containerd (from the plugins/containerd module - see "Module layout" below). It then writes the generated containerd config fragment and RuntimeClass manifest.

containerd must use config version 2 and import the generated fragment:

version = 2
imports = ["/etc/containerd/conf.d/*.toml"]

Restart containerd only after validating the merged configuration:

sudo containerd config dump
sudo systemctl restart containerd
kubectl apply -f /etc/containerd/conf.d/platform-factory-runtimeclass.yaml

Select the VM runtime per Pod:

spec:
  runtimeClassName: platform-factory

Omit runtimeClassName to retain the cluster's normal OCI/container runtime. The generated fragment can be inspected without installing anything:

go run ./plugins/containerd/cmd/platform-factory-containerd config
go run ./plugins/containerd/cmd/platform-factory-containerd runtimeclass

The generated fragment selects sandboxer = "shim" (not the default "podsandbox") and runtime_type = "io.containerd.platform-factory.v1" - the two settings that route pod sandbox creation to platform-factory-shim instead of containerd's default runc-based sandbox model.

Module layout

plugins/containerd is its own Go module (plugins/containerd/go.mod), tied to the main module only through go.work for local development and through platform-factory-runtime's CLI contract at runtime - it does not import any internal/ package. This mirrors the project's rule that every runtime-engine integration beyond the microVM (native) and OCI interfaces themselves is an out-of-module plugin, and that internal/ is never a plugin boundary even where Go's own visibility rule would technically allow it (a plugin module sharing the repo's import path prefix can still see into internal/): the main module never depends on a plugin, and a plugin only ever consumes a public surface - either the main module's CLI, like plugins/containerd consuming platform-factory-runtime, or a stable Go contract under api/, like plugins/kubevirt consuming api/microvm (Spec, ValidateCommon, Forward) instead of the internal/microvm and internal/networking packages that actually implement the native backend. internal/microvm.Spec and internal/networking.Forward/ParseForward are now aliases of their api/microvm originals for exactly this reason: one definition, consumed identically whether the caller is in-module or a plugin. plugins/kubevirt (see the KubeVirt backend in platform-factory microvm --backend=kubevirt, driven by plugins/kubevirt/cmd/platform-factory-kubevirt) follows the same shape as plugins/containerd at the module level, just through a different kind of public surface.

CI coverage

CI validates this contract on a kind cluster containing one control-plane and two workers. Two ordinary Pods become Ready across distinct workers. Two Pods selecting platform-factory are also scheduled across distinct workers, but fail closed at sandbox creation because the disposable kind nodes deliberately have neither the runtime installed nor /dev/kvm. This proves CRI handler selection and scheduler behavior; it is not reported as a MicroVM boot test. Native KVM boot remains covered separately by the MicroVM jobs, and the shim's own Sandbox/Task TTRPC contract is covered by plugins/containerd's unit tests plus a local (non-CI) end-to-end validation against a real containerd and crictl.

A separate multi-node job stops one kind worker container after placing two ordinary-runtime replicas across both workers. Kubernetes must mark the lost node NotReady, evict its Pod under an explicit five-second toleration and restore both replicas on the surviving worker. This proves control-plane scheduling and worker-loss recovery, not MicroVM migration or restart.

Current boundary

Image pulling and OCI bundle construction remain containerd/CRI duties. The secure runtime accepts the strict OCI subset documented by platform-factory-runtime features; host bind mounts, terminals, hooks, cgroups and Linux namespace requests fail closed - the shim does not, and cannot, loosen that policy on the runtime's behalf. platform-factory-shim's Task service does not implement exec, pause/resume, checkpoint, or stats: a MicroVM-backed container is not a process tree those operations apply to, and it fails them closed rather than silently no-op.