feat(os): give both guest kernels the tc and checkpoint/restore capabilities Incus needs

[kvinwang]

Problem

Two guest-kernel capability gaps that Incus system containers inside a CVM run into, reported against the 0.5.9 guest (6.9.0-dstack):

Per-instance bandwidth limits (#1176). Incus implements limits.max / limits.ingress / limits.egress on a bridged NIC as an HTB root qdisc and class on the host-side veth for egress, and an ingress qdisc with a u32 filter and a police action for ingress. That needs NET_SCH_HTB, NET_SCH_INGRESS, NET_CLS_U32 and NET_ACT_POLICE. NET_SCHED alone provides none of them — each is its own tristate — so the manager accepts the setting and then fails to start the instance:

Failed to create root tc qdisc ... qdisc htb ... Specified qdisc kind is unknown

Both current backends lack all four: the locally built Yocto 6.18 kernel-config has them (and NET_CLS_ACT) as not set, and x86_64_defconfig plus os/mkosi/components/kernel/kernel.config produces the same.

Seccomp notification handling (#1180). LXC retitles its monitor process to [lxc monitor] <path> <name> with prctl(PR_SET_MM, PR_SET_MM_MAP, ...), and Incus's seccomp handler resolves a monitor PID to its instance by that title. The kernel only implements PR_SET_MM_MAP under CONFIG_CHECKPOINT_RESTORE; without it the prctl returns EINVAL, the monitor keeps its incusd forkstart command line, incusd logs Failed to find container for monitor <pid>, and with seccomp_listener_continue every intercepted syscall (security.syscalls.intercept.*) is resumed unhandled instead of getting the container-aware answer. Here the two backends differ: the Yocto 6.18 build already has the symbol, through the cfg/lxc.scc and cfg/criu.scc that meta-virtualization adds to KERNEL_FEATURES (the 0.5.9 kernel did not), while x86_64_defconfig leaves it at its Kconfig default of n, so the mkosi kernel is missing it and nothing asserted it on either side.

No gate would have caught either. Neither Incus nor anyone else ships a kernel-config checker for Incus; the closest thing is LXC's own lxc-checkconfig, which Incus's requirements page implicitly defers to ("any kernel feature required by the LXC version in use"). Run against the mkosi kernel it reports seven symbols missing — CHECKPOINT_RESTORE, MACVLAN, NETFILTER_XT_MATCH_COMMENT and the four socket-diag interfaces — all of which the Yocto kernel has. It does not know about traffic control, so it would not have caught #1176.

Fix

1. tc pieces (b215bdb): NET_SCHED, NET_CLS_ACT, NET_SCH_HTB, NET_SCH_INGRESS, NET_CLS_U32, NET_ACT_POLICE, all =y, in both fragments. Built in rather than =m for the same reason as CONFIG_TLS: no kernel-module-* package to forget in the Yocto rootfs (a missing one fails at runtime, not at build time), and no reliance on module autoload from inside the manager's own namespaces. ~100 KB of text in total.
2. CHECKPOINT_RESTORE=y (9be802d) in both fragments — new on mkosi, an assertion on Yocto so neither backend can lose it.
3. The rest of what lxc-checkconfig asks for (ecae1f8): MACVLAN=y (Incus nictype=macvlan), NETFILTER_XT_MATCH_COMMENT=m (-m comment on the xtables frontends; the Yocto rootfs already ships kernel-module-xt-comment, mkosi installs every module it builds), UNIX_DIAG/INET_DIAG/PACKET_DIAG/NETLINK_DIAG=y (ss, CRIU). Values match what the Yocto tree already had.
4. Gate both kernels on lxc-checkconfig (bb3acfc): vendored verbatim at os/common/scripts/lxc-checkconfig (lxc/lxc 969a3c5, LGPL-2.1-or-later, license text added under LICENSES/), driven by check-lxc-kernel-config.sh, which runs it with CONFIG=<built .config>, fails on any missing/required line, ignores the script's host-state probes (cgroup mounts, /proc/self/ns/cgroup) and refuses to pass if fewer than 20 symbols came back enabled. The only exemptions are IP_NF_TARGET_MASQUERADE / IP6_NF_TARGET_MASQUERADE, backwards-compat aliases that just select NETFILTER_XT_TARGET_MASQUERADE (which both kernels build); the IPv6 one is unreachable on mkosi anyway (needs legacy ip6tables). Called right after check-kernel-config.sh in kernel-build.sh (mkosi) and export-artifacts.sh (Yocto); acceptance.sh checks the mkosi wiring.

All new fragment lines live in one block that is byte-identical between the two backends, following the bridge-filtering block. parity.json asserts the new symbols on the installed mkosi kernel config.

Security assessment of CHECKPOINT_RESTORE (asked for in #1180). On x86 the symbol gates: PR_SET_MM (needs CAP_CHECKPOINT_RESTORE or CAP_SYS_ADMIN in the caller's user namespace), the kernel.ns_last_pid sysctl and the IPC *_next_id sysctls (same check), PTRACE_O_SUSPEND_SECCOMP (init-namespace CAP_SYS_ADMIN), PTRACE_SECCOMP_GET_FILTER / GET_METADATA on a ptraced task (CAP_SYS_ADMIN), MSG_COPY on msgrcv (non-destructive read of a queue the caller can already read) and the ARCH_MAP_VDSO_* arch_prctl. It also selects PROC_CHILDREN and KCMP (KCMP was already y on both backends). Every path is capability- or ptrace-gated guest-userspace surface; none of it faces the host, so it does not change what a malicious host can reach, and it is what every general-purpose distro kernel ships (Debian, Ubuntu, Fedora all set it; docker checkpoint / CRIU need it). The CVM is single-tenant, so intra-guest process introspection is not a boundary dstack defends — and the Yocto image has been shipping it since it moved to 6.18.

docs/guest-netfilter-capabilities.md records all of the above next to the netfilter matrix (and corrects that table's "Since" column: the netfilter work is in v0.6.0-rc0, so it ships in 0.6.0, not 0.6.1).

Verification

• mkosi backend: ran the kernel-config stage of kernel-build.sh locally on linux-6.18.40 (both dstack patches applied): x86_64_defconfig → merge_config.sh with the final fragment → olddefconfig; check-kernel-config.sh and check-lxc-kernel-config.sh both pass. Resulting .config has every requested symbol at the requested value, plus KCMP=y, PROC_CHILDREN=y, NET_XGRESS=y.
• Yocto backend: merged the final dstack-docker.cfg into the deployed kernel-config of a local linux-yocto 6.18.39 build with merge_config.sh + olddefconfig against the bitbake kernel source; both gates pass. The diff against the previous config shows exactly the requested symbols flipping (plus NETFILTER_SKIP_EGRESS=y, a def_bool that follows NET_XGRESS).
• The new gate fails on the old kernels: against next's mkosi config it reports the seven symbols above and exits 1; against an empty file it exits 1 with reported only 2 enabled symbols; refusing to proceed. On the new configs it counts 30 enabled symbols.
• os/mkosi/tests/acceptance.sh passes, shellcheck is clean on the wrapper, reuse lint lists LGPL-2.1-or-later as used with no unused licenses.
• Not done here: a boot test of a candidate image with Incus. The issue reporter offered to rerun the Incus limits.max / traffic-measurement and sysinfo interception gates on a disposable CVM once a candidate image exists; that is the right place for the end-to-end check.

Closes #1176
Closes #1180

[Copilot]

🟢 Approval recommended

The changes are limited to kernel config fragments/parity/docs, are consistent across both backends, and align with the stated Incus/LXC requirements and existing capability-matrix conventions.

Pull request overview

This PR closes two kernel capability gaps that prevent Incus system containers from working correctly inside a CVM, by enabling required traffic-control (tc) primitives and checkpoint/restore support on both guest OS backends (mkosi + Yocto) and documenting the new guarantees.

Changes:

• Enable HTB/ingress/u32/police tc features (NET_SCH_HTB, NET_SCH_INGRESS, NET_CLS_U32, NET_ACT_POLICE) built-in for Incus per-instance bandwidth limits.
• Enable CONFIG_CHECKPOINT_RESTORE=y to support LXC’s PR_SET_MM_MAP usage needed by Incus seccomp notification handling.
• Extend mkosi parity assertions and document these non-netfilter kernel capabilities in the guest capability matrix.

File summaries

File	Description
os/yocto/layers/meta-dstack/recipes-kernel/linux/files/dstack-docker.cfg	Adds built-in tc + checkpoint/restore kernel symbols for the Yocto guest kernel fragment.
os/mkosi/parity.json	Asserts the new kernel config requirements for mkosi parity gating.
os/mkosi/components/kernel/kernel.config	Adds the same built-in tc + checkpoint/restore kernel symbols for the mkosi guest kernel fragment.
docs/guest-netfilter-capabilities.md	Documents tc and checkpoint/restore capabilities alongside the existing nested-manager capability guidance.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.