Skip to content

mklinux v7.0-mk5

Latest

Choose a tag to compare

@congwang-mk congwang-mk released this 17 Nov 18:54
· 4 commits to master since this release

Fifth mklinux release, 114 patches on top of v7.0-mk4.

Highlights

  • arm64 support. Multikernel now runs on arm64 as well as x86_64: spawn kernels start through PSCI, share the GICv3 with the host as tenants, and get their MSIs through the host's ITS. This is also the first release with prebuilt arm64 packages.
  • Lent devices are confined by the IOMMU. On both architectures, a PCI device lent to an instance can only reach that instance's memory, is held by a stub driver so the host cannot bind it again, and is reset when it comes back.
  • Spawns work behind VT-d interrupt remapping. The host lends each spawn its own remapping entries and checks what the spawn writes into them, and a successor can take over the VT-d units of a fenced host without turning translation off.
  • Takeover can be chained. A kernel that took the machine over from its host becomes instance 0, can use the adopted memory as kernel memory, and can arm and hand over to a successor of its own, any number of times. On x86 this rests on a new per-CPU park table that any kernel can use to find and wake a parked CPU.
  • A dying spawn no longer disturbs the others. Each parent/child pair has its own message rings, and on arm64 a force halt through SDEI reaches a CPU even with every exception masked.
  • A spawn can show its console on the host's screen, for machines reached only through a BMC's remote KVM.

Important

ABI change: the multikernel flag and instance id of kexec_file_load() moved to the upper half of the flags (KEXEC_FILE_MULTIKERNEL, KEXEC_FILE_MK_ID_MASK). A loader built against v7.0-mk4 or earlier must be rebuilt.

Changes

arm64: multikernel runs on arm64. CPUs are spawned, confirmed off and released through PSCI, an Image is loaded into an instance with kexec_file_load(), and the instance boots from a device tree the host builds for it. The doorbell is an SGI sent by MPIDR. A spawn is a tenant of the GICv3: it leaves the distributor and LPI setup to the host and initialises only the redistributors and CPU interfaces of its own CPUs. An instance gets PCI devices only, no platform devices, and their MSIs are routed through whichever host ITS serves the device. A force halt uses SDEI event 0 where firmware has it, which reaches a CPU with every exception masked, and the stop IPI otherwise. Memory hot-remove on arm64 checks the requested range, splits or refuses a partial removal of a linear map leaf, frees the page tables and vmemmap it empties, and can map part of removed memory back.

Device leases: what the host sets up for an instance's use of a PCI function is gathered into a lease, which is the only record of a lent function. A function the host lets go of is held by a stub driver, mk_pciback, so no host driver can bind it again while an instance drives it. The DMA of lent devices is confined to the instance's memory through the IOMMU, on arm64 and x86, and follows the memory grant of a running instance. A function is reset when its lease ends, while its DMA is still confined, and a halted instance's devices stay blocked until its CPUs are off. An instance can write only the capabilities that run its function. pcifront is the platform driver of a spawn's PCI buses, so their DMA is configured from the tree, and a spawn lends only the devices none of its own drivers has.

VT-d: the host leaves alone the units another kernel drives, lends blocks of interrupt remapping entries to a spawn and checks every entry the spawn composes, and a spawn remaps its interrupts behind a unit of its host. A spawn that took over can claim the units of its fenced host with iommu-add, with translation on throughout.

Takeover: a successor fences its host one CPU at a time, confirms each CPU parked by asking it, stops talking to the host before force halting it, and then takes ID 0 as a kernel booted on the machine has it. On arm64 the fence turns the host's CPUs off, through SDEI first, and the successor claims the host's ITSes; the GIC's LPI tables are kept out of its reach and preserved memory is passed on to every later kernel. Room is made for the host's RAM before memory is laid out, adopted memory marked normal is onlined as kernel memory, and a baseline request can claim memory no kernel runs on, so a successor can arm a successor of its own. The host's functions are adopted without a bus of the spawn's own.

x86 park table: parked CPUs no longer wait on slots shared between kernels. The first kernel builds a table with one cell per CPU of the machine. A CPU parks in its own cell and claims a command there in a single word, so any kernel can find and wake it across any number of takeovers. Each kernel has its own trampoline page, and the first kernel's pool CPUs start from their cells too.

Messaging: each parent/child pair has its own IPI rings, so a child halted mid-send can no longer cut the other children off from the parent. Handlers are told which kernel sent a message, messages go only to a running kernel, and sending stops to an instance that did not answer.

kexec: the multikernel flag and instance id move to the upper half of the kexec_file_load() flags, as KEXEC_FILE_MULTIKERNEL and KEXEC_FILE_MK_ID_MASK, where they no longer collide with KEXEC_FILE_NO_CMA and KEXEC_FILE_FORCE_DTB. This changes the ABI: a loader built against the old values must be rebuilt. Spawn kernels are no longer handed the host's KHO state, and kexec segments are placed within their bounds.

Console: a spawn can be handed the screen its host is seen on as a simple-framebuffer, for machines reached only through a BMC's remote KVM.

Fixes: a removed region's chunk is taken out of the instance pool (another genalloc BUG), memory hotplug onlines every section a subsection range touches, park descriptors no longer split-lock, a late RST no longer closes a listener in mk_transport and late connect answers are handled as virtio does, pooled devices are named by label with device aliases dropped, and every reg entry of a host tree node is copied on arm64.

Full changelog: v7.0-mk4...v7.0-mk5

Install

Packages built from this tag on GitHub Actions with the Ubuntu 26.04 generic config plus CONFIG_MULTIKERNEL=y, without debug info (so no BTF). They are not Secure Boot signed; modules are signed with a per-build key. Verify downloads with sha256sum -c --ignore-missing SHA256SUMS.

Debian/Ubuntu (amd64):

sudo apt install ./linux-image-7.0.0-mk5_7.0.0-1_amd64.deb ./linux-headers-7.0.0-mk5_7.0.0-1_amd64.deb

Debian/Ubuntu (arm64):

sudo apt install ./linux-image-7.0.0-mk5_7.0.0-1_arm64.deb ./linux-headers-7.0.0-mk5_7.0.0-1_arm64.deb

Fedora/RHEL (x86_64):

sudo dnf install ./kernel-7.0.0_mk5-2.x86_64.rpm ./kernel-devel-7.0.0_mk5-2.x86_64.rpm

Fedora/RHEL (aarch64):

sudo dnf install ./kernel-7.0.0_mk5-2.aarch64.rpm ./kernel-devel-7.0.0_mk5-2.aarch64.rpm