v2.0.0-beta.5

The first public release. Restore in a Kubernetes pod with the same
warmctl up NAME, multi-GPU engines restore under NVIDIA driver 610, and a
cold restore from a file share reads the snapshot in parallel (GLM-5.3-Flash
on 8×H100: 75 s → 57 s).

Kubernetes (prototype)

• up picks where to restore. In the warmctl runtime image (marker
  /opt/warmctl/RUNTIME) it restores in place, in its own container; otherwise
  it asks Docker for a receiver container. With neither, it stops with an
  error. --mode host|in-place overrides the choice.
• In place, warmctl prepares its privileged container as a receiver: tmpfs at
  /tmp and /dev/shm, the snapshot's extension and cuInterpose mounted
  read-only, the GPUs the device plugin assigned. The agent runs in a UTS
  namespace of its own with the host name the engine had. The pod needs no
  volumes; a node-local /var/lib/warmctl only keeps pulled snapshots.
• The preflight refuses hostNetwork and hostPID, a missing capability or
  seccomp profile, a read-only /proc/sys and an image built from another
  vLLM base. warmctl empties /workspace and its run directory only when they
  are empty or marked by an earlier restore; foreign data stops the restore
  untouched.
• The gateway listens on :8080 from the start: /health answers 503 while
  pulling and restoring and 200 after the canary, so a readinessProbe keeps
  traffic away until the engine is warm.
• The runtime image is registry.warmctl.io/warmctl/runtime, a tag per
  release and :latest; make pod-image builds it for another registry or
  vLLM base. deploy/kubernetes holds its Dockerfile, a Deployment and
  Service, a node-preparation DaemonSet and a KEDA ScaledObject on
  vllm:num_requests_waiting. Design and open risks:
  docs/kubernetes.md.
• Tested on CPU in pod-like containers (make e2e-test), not yet on a GPU
  node of a Kubernetes cluster.

Driver 610

• Multi-GPU engines under driver branch 595 and later run on the host's
  libcuda: build puts the host's library directory first in
  LD_LIBRARY_PATH, because the vLLM image's forward-compatible libcuda
  cannot checkpoint processes that share GPU memory under a 610 kernel
  module. Single-GPU engines keep the image's libcuda.
• Published for 610.57.04: eight single-GPU H100 snapshots, gpt-oss-120b on
  2×H100 and GLM-5.3-Flash on 8×H100.

Restore from a file share

• up reads the snapshot ahead from the start of the restore: the CRIU
  images first, then the weights round-robin across ranks, with 32 parallel
  readers capped by available memory (WARMCTL_PREFETCH_WORKERS overrides).
  Each range is also handed to the kernel with posix_fadvise(WILLNEED), and
  up logs how the prefetch went. GLM-5.3-Flash on 8×H100 restored cold from
  a virtiofs share in 56.7 s instead of 75.1 s; a warm restore stays at
  40–44 s.

Known issues

• Under driver 610, MiMo-V2.6-Flash and DeepSeek-V4.1-Flash do not start on
  the host's libcuda: their kernels ship as CUDA 13.4 PTX, which needs a newer
  libcuda to compile.
• The first restore right after a host boots can fail in CRIU (the restored
  root task is killed after about a second); later restores pass.
• warmctl pull onto a virtiofs share writes at about 20 MB/s.
• GLM-5.3-Flash at TP4 on B200 fails in a Triton kernel of its linear
  attention under 32 users with 4K random-token prompts; TP8 and ordinary
  text work.

Install: unpack warmctl-v2.0.0-beta.5-linux-amd64.tar.gz and keep
warmctl-linux-amd64 and cuinterpose/ side by side (see the README's
quickstart). The Kubernetes runtime image is
registry.warmctl.io/warmctl/runtime:v2.0.0-beta.5.

sha256 5f67e717ca3ca6b3b2d10b9e7d9692210872f0383200ca77e97e27cf9d676d22 warmctl-v2.0.0-beta.5-linux-amd64.tar.gz