k3sm v0.1.7

vm Pods answer on their own pod IP from anywhere in the cluster, and two Macs can run the control plane together as an experimental preview.

Added

• vm Pods answer on their pod IP from every node. The node a vm Pod runs on holds its pod IP and relays each TCP connection to the guest, so another node, a native Pod on the same node and the node itself reach the Pod at the address kubectl get pods -o wide shows, and Services reach vm Pods across nodes. The relay covers the ports the Pod declares as a containerPort and the ports an EndpointSlice targets at that Pod. Other ports, and UDP to a vm Pod's IP, are not relayed. The root network helper binds ports below 1024 on that address only for those same ports, decided from its own reads of the cluster.

Experimental

• Two Macs can share the control plane. sudo k3sm install --cluster-init forms an embedded etcd control plane on the first server and sudo k3sm install --server-join adds a second. Each server's pod range is the /24 its --mesh-ip names. A join whose range is already held is refused before anything is added to etcd, and a server's name stays bound to its Mac. Two servers tolerate no failures: losing either one stops writes until it returns or the survivor runs k3sm server --cluster-reset. Workers stay attached to the server they joined through. A controller manager or scheduler that loses its leader lease when quorum is lost is counted as a crash, so repeated quorum loss can park a server. See high availability.

k3sm v0.1.6

Hosts survive heavy pod-network traffic, shell tools see mounted paths everywhere, and Pods get curl, TLS, eviction, logs on disk and re-attach across a daemon restart.

Breaking

• Runtime proto: PodBox.rootfs_path is removed. Anyone building against the runtime gRPC contract must stop sending the field and regenerate. The daemon now derives the root filesystem path itself and ignores a caller-supplied one. Nothing changes for a Pod author or for the shipped k3sm binary.
• External-datastore HA is removed, and a multi-server control plane is not available in this release. The k3sm server flags --datastore-endpoint and --datastore-endpoint-file, and their environment variable, are gone, and an old argument list now exits on an unknown flag instead of quietly falling back to SQLite. There is no conversion from an external database to the new HA datastore. Single-node installs are unchanged and keep the SQLite-backed datastore.

Added

• A defence against the host kernel panic seen on the pod network. Every pod address is a loopback alias, and a connection that outlived its alias could re-route onto the mesh interface and trip a macOS kernel panic from an ordinary send. TCP connections on the pod network now have their segment size lowered after connect, and a torn-down pod's address is blackholed so an open connection fails cleanly instead of re-routing.
• Shell tools see mounted paths. The re-signed shadow set now covers tar and the usual coreutils, and the path shim also rebases directory and metadata calls (mkdir, rmdir, rm, mv, chmod, ln, touch, cp) and recursive walks (rm -r, ls -R, chmod -R, find, cp -R). kubectl cp into and out of a mounted path works. sudo k3sm install lays the set down.
• curl and TLS clients work in native Pods. Pods may read the system TLS configuration, so the stock curl and anything linked against the system SSL library no longer abort at startup before they dial.
• Node identity under the Node authorizer. The server's in-process node now runs as system:node:<name> instead of the admin identity, so a node bug can no longer act as cluster admin. The set of RBAC objects k3sm creates is pinned by a golden test.
• Node-pressure eviction. When memory runs low the node marks MemoryPressure and evicts one Pod at a time, ranked as the kubelet ranks them, with the usual Evicted Event and DisruptionTarget condition. After three evictions in five minutes it halts and says so on the node.
• k3sm install --data-volume. Puts the data root on a dedicated, size-capped APFS volume instead of a plain directory on the boot disk. It creates a case-sensitive volume in the boot container (or adopts one you already declared in /etc/fstab), mounts it at /var/lib/k3sm hidden from Finder, Spotlight and Time Machine, and migrates any existing data root onto it, verifying the copy before the old tree is renamed aside. --data-volume-size sets the quota (100 GiB by default, a floor of 32 GiB), and --data-volume-encrypt protects it with a random passphrase kept in the System keychain.
• k3sm datavol mount|status|delete. mount is what the new io.k3sm.datavol LaunchDaemon runs at boot and is safe to run by hand; status reports the volume, its quota, its usage and any leftover pre-migration copy with no privilege needed; delete --yes destroys the volume and every declaration of it.
• k3sm status reports the data volume and more. The data-root row names the volume, its quota and its usage, and warns once usage passes 90% of the quota. A datavol row tracks the boot mount daemon, and a pre-volume row appears while a migration's pre-migration copy is still on disk. Status also flags Pods stuck terminating, prints the escape, reports a worker's agent daemon and its credential, reports drift in the shadow set, and renders k3sm doctor on the status record with --report. A worker no longer shows control-plane rows.
• Container logs on disk. The runtime writes container logs under /var/log/pods in the CRI format, and kubectl logs reads them as the kubelet does, with rotation and cleanup. A vm container's status names its log file.
• A resident shim keeps container output and exit status. Native containers keep stdio and exit status in a small resident process, so output written just before exit reaches the log. Exec and teardown work on Pods that run under it.
• Pods survive a node-daemon restart. A restarted daemon re-attaches live Pods instead of recreating them, and restarts a single container of a re-attached Pod in place.
• Image pull failures read like the kubelet's. A failed pull shows as a retryable waiting state (ErrImagePull, ImagePullBackOff) instead of a failed container.
• Per-container stop and ephemeral containers. A single container can be stopped terminally, and ephemeral containers and the Pod-level runAsNonRoot are mapped.
• Projected volumes refresh. ConfigMap, Secret and projected volumes update at the kubelet's cadence with an atomic swap, so a running Pod sees new content.
• Helm charts. HelmChart and HelmChartConfig objects in helm.k3sm.io/v1 install and uninstall charts, with the same fields as k3s.
• Operator manifests. Manifests placed in a root-owned directory are applied to the cluster automatically.
• Secrets encryption at rest. sudo k3sm install --secrets-encryption generates a key on the Mac and encrypts Secrets in the datastore of a new cluster. It is opt-in.
• sudo k3sm uninstall --purge --yes. Removes what a plain uninstall keeps: the cluster data, the service user, the data volume and the kubeconfig context.
• Worker lifecycle. A joining Mac gets its own agent LaunchDaemon, reuses its stored node credential on restart, and deregisters from the cluster on uninstall. --mesh-ip lets the installer own the server's mesh address.
• Ingress, load balancers and service policy. The ingress binds through the network helper and publishes its endpoints. loadBalancerSourceRanges is enforced on LoadBalancer traffic. A Service published on a port the node keeps private is rejected. Pods get namespace service links in their environment.
• MLX scheduling. MLXModel derives GPU slots and a cumulative memory fit from the ceiling and reports replica counts in status.
• Faster proxy paths. The UDP relay and the Service proxy pick routes without locks, and the per-query and per-connection allocation counts dropped.

Experimental

• Groundwork for Thunderbolt direct links between Macs. The contracts (MeshPeer endpoint candidates, the net.k3sm.io/v1alpha1 DirectLink type, the derived link addresses) and the networking library and root-helper verbs landed; nothing is wired into the node in this release, so no link is enumerated or used yet and no DirectLink object appears. The API is alpha.
• Sharded MLX models (alpha, trusted tenancy only). An MLXModel may set spec.distributed (ranks, backend, parallelism); the operator places the ranks as gang-scheduled rank Pods with per-rank DNS and reports placement and link health in status. In this release the ring backend places over the existing mesh, because direct links are not yet wired into the node; the jaccl backend needs RDMA-capable direct links and reports ShardsPlaced=False until they exist. Not yet run on hardware.

Changed

• Native Pods refuse emptyDir medium Memory. A native Pod that asks for a memory-backed emptyDir is refused with a clear error instead of silently getting a disk-backed one.
• Workers do not enforce NetworkPolicy. A joined worker resolves policy against its own Pods only and enforces nothing for traffic it cannot attribute. Policies are enforced for Pods on the server node. The limitations page lists the ceiling.
• A mid-install failure leaves the old install intact. The install root is staged and swapped in atomically, and the staged inputs are verified before anything is written.
• The data-root refusal message now names sudo k3sm datavol mount as the remedy for a declared but unmounted data root.

Fixed

• Credentials and logs are locked down. The admin token and the datastore connection string no longer appear on the server's command line, the mesh keys live under a root-owned state root, and the daemon log directory is restricted to root and admin.
• Installs refuse unsafe paths. The installer refuses a symlink where it would change ownership, an untrusted launcher directory and a join to the control plane's own node name, and says how to fix each.
• Worker joins are sturdier. A join waits for the network helper and for a recovered start before judging failure, preflights the join endpoint and its CA, releases a mesh allocation when the join fails, and issues the node certificate for the assigned mesh address. Anonymous join requests and attempts per token are bounded.
• Mesh teardown and resume. The mesh is torn down on exit, a busy join listener is retried, a resumed peer seed advances correctly, and MeshPeer events coalesce into one reconcile.
• A control plane that fails to come up says why. A component that dies before it is marked supervised is reported, bring-up failures are recorded on the crash-loop breaker with secrets redacted from the log tail, and a toolchain-less control-plane build parks on first failure.
• Clean exit. The node stops the runtime, its Linux VMs and the control plane together when it exits.
• Linux guests and vm Pods. Probes dial a vm Pod at the guest's live address, PodReady waits for its transport, and guest networking waits for carrier and retransmits its address request.
• Sandbox hardening. Pod profiles refuse a per-IP host in a network filter, refuse symlinked path components on every vm path, deny the daemon's private trees and loopback dials to the node's private ports, and are swept when stale...

v0.1.5

A Pod can build with the Mac's Xcode toolchain, and a control plane that dies now says so.

Added

• k3sm.io/xcode-toolchain does what it says. A Pod carrying the annotation gets read
  access to the compilers, linker and SDKs in the node's Xcode developer directory. The
  annotation existed before this release but did not deliver that, in two different ways
  depending on the machine: on a Mac with only the Command Line Tools installed an annotated
  Pod failed to start at all, because the node offered a toolchain directory the sandbox
  generator refuses and so no profile was produced; on a Mac with full Xcode selected the Pod
  ran and the grant was written, but nothing told the Pod to use it, so the toolchain lookup
  inside the sandbox fell back to the Command Line Tools and the build quietly used those.
  Both are fixed, and where the grant applies a Pod now builds against the toolchain it was
  given. It is opt-in per Pod and it is read access: the compilers, the linker, the SDKs
  and xcrun lookups work, while driving a full xcodebuild build is out of reach of any
  read grant. A node with only the Command Line Tools is granted nothing and needs nothing —
  that toolchain is already reachable from inside a Pod — and an annotated Pod there now runs
  normally and records a warning Event saying it got no grant. The developer directory is read
  once, when the node daemon starts, so sudo xcode-select -s ... takes effect on the next
  restart.
• A toolchain row in k3sm doctor. It reports which of those cases the node is in and
  what to run if you want the other one. k3sm.io/xcode-toolchain is now written up in the
  Limitations page, including the harmless xcrun cache message that appears on every
  invocation inside a Pod and cannot be switched off.
• A crash-loop circuit breaker for the control plane. Reporting a dead component (below)
  turns a persistent fault into an unthrottled restart loop, so the daemon now keeps a crash
  record beside its work directory. Five crashes within ten minutes trips the breaker, and it
  stays tripped until you clear it with k3sm server --clear-crashloop or delete the file. A
  cluster that cannot come up stops and says why rather than cycling.

Fixed

• A control-plane child that died was never noticed. Nothing watched the control plane
  once it was up, so a kube-apiserver, kine, kube-scheduler or kube-controller-manager
  that died hours later left k3sm server running — which meant launchd's restart policy
  never fired and the cluster sat there broken. A component that dies is now reported and the
  daemon exits, so launchd restarts it.
• k3sm dev up hung forever against a wedged apiserver, logging nothing. Its readiness
  probes were handed a context with no deadline, so the timeouts written beside them could
  never fire. They fire now.
• A data race between the Pod-status backstop and the Virtual Kubelet Pod workers, which
  ran every ten seconds on every node.
• The process reaper wakes when its work is cancelled instead of polling for it, and the
  Service proxy copies spliced TCP through a pooled buffer.

Also in this release

An experimental helper that runs a container inside its own lightweight Linux VM, built on
Apple's open-source Containerization package, is present as a spike. No RuntimeClass selects
it and nothing uses it yet.

Full documentation: https://k3sm.io/docs/

v0.1.4

One command that says what is running, and daemons that refuse to write into an unmounted data
root.

Added

• k3sm status. One screen that answers "is my cluster up, and if not, which piece is down
  and why": a one-line verdict first, then one row per component — install, netd, server,
  apiserver, node, workloads, data root, datastore, kubeconfig — each with its state, a detail,
  and the command that fixes it. k3sm status daemons shows the launchd view (state, pid, runs,
  last exit, plist, log); k3sm status cluster the apiserver, node, Pods by phase, control-plane
  children and Linux guest hosts; k3sm status logs [netd|server] tails the daemon logs. -o json
  emits the same report for scripts, --wait blocks until the cluster is running, --watch
  re-renders in a terminal. The exit code is the verdict: 0 running, 3 stopped, 4 degraded, 5 not
  installed, 6 unknown (k3sm status --help is the reference). Colour and glyphs follow NO_COLOR
  and whether stdout is a terminal; the text is never the only signal.
• Data-root guard. A data root kept on its own volume is declared in /etc/fstab; when that
  volume is not mounted, the netd helper, the server and sudo k3sm install now refuse to start
  or install rather than writing a shadow directory into the empty mountpoint — the message names
  the mount command. On a plain-directory data root, the netd helper realigns a root-owned data
  root to the service user on every start, so a wrong owner is repaired by restarting it.

Fixed

• A data root that was not mounted at boot no longer leaves the server crash-looping on
  permission denied behind a root-owned shadow directory, and can no longer receive a fresh,
  empty datastore over the real one. k3sm status names the state and the fix.

Full documentation: https://k3sm.io/docs/

v0.1.3

The k3sm command is on PATH after install, and k3sm kubectl works without sudo.

Added

• A k3sm launcher on PATH — sudo k3sm install links /usr/local/bin/k3sm to the installed binary, so every new terminal finds k3sm with no profile edits. The link is a symlink, never a copy, so it always runs the same binary the daemons do. The installer refuses to replace a non-symlink already at that path and refuses to link into a directory that is not root-owned or is group/other-writable, naming the fix in each case; sudo k3sm uninstall removes only the link it created.

Fixed

• k3sm kubectl and k3sm kubeconfig work as you, not just as root. Both verbs looked only in the control plane's private work directory for the admin kubeconfig and the bundled kubectl, so after a normal install they failed for the user who ran it. They now use the k3sm context that install merges into your ~/.kube/config (or $KUBECONFIG) and the kubectl installed alongside k3sm; a user-supplied --context still wins, and K3SM_WORK_DIR still pins a non-default server work directory.
• The installer's closing hint now tells the truth for the current shell: when k3sm is not yet on PATH it names the launcher and the two ways to reach it (a new terminal, or adding /usr/local/bin to PATH) instead of suggesting a command that could not work. The pre-escalation banner lists the symlink alongside everything else the install lays down.

Full documentation: https://k3sm.io/docs/

v0.1.2

One-command in-cluster image builds, one cluster address for the node registry, and self-recovering daemons.

Added

• One-command builds — k3sm build builds any Dockerfile: a copy-only recipe packages natively in about a second, and a recipe with RUN steps builds on the cluster's build engine, which starts automatically on first use. The image is recorded in the node's image store under its tag either way, ready for a Pod to name; --output additionally writes a portable artifact.
• Build and push in one step — k3sm build --push publishes the built image after the store recording; a bare tag publishes to the node's own registry, a full reference pushes as written.
• Linux images as a first-class build target — k3sm build --platform linux/arm64 builds a Linux image even from a copy-only Dockerfile; multi-platform builds export and push a full OCI index.
• A raw buildx surface for the engine — k3sm builder buildx drives the build engine with the bundled, digest-verified buildx; k3sm builder delete fully resets the engine, cache included.
• One cluster address for the node registry — each node publishes a registry-<node> Service backed by its relay address, so in-pod tools — Linux guests included — reach the registry at one name, and the registry hosting ConfigMap now carries hostFromClusterNetwork.
• Bare image names — a Pod naming app:v1 resolves from the node's registry first, then from cluster peers, before Docker Hub; no registry prefix required for images you built or pushed locally.

Fixed

• sudo k3sm install over a running node now sequences the daemon restart around launchd's asynchronous teardown, retries transient bootstrap failures, and verifies both daemons serve before reporting success — the in-place upgrade path is exercised by the release suite on every cut.
• The installer refuses a k3sm-vmhost helper that lacks the virtualization entitlement, naming the exact fix, instead of installing it and leaving every vm Pod Pending on an opaque scheduling message.
• The netd helper exits and restarts cleanly if its control socket is removed or replaced; the cluster-DNS listener retries its bind indefinitely instead of giving up; a failed Service-watch start no longer leaks a stale retry loop.
• The build engine's output no longer prints a Docker Desktop deep link.

Full documentation: https://k3sm.io/docs/

v0.1.1

Patch release. Interactive Linux workloads, a node-local image registry, and in-cluster image builds.

Added

• Interactive terminals for Linux (vm) Pods. kubectl exec -it allocates a real pseudo-terminal in the guest (it was refused before); terminal resize delivers SIGWINCH; every vm container now gets a minimal standard /dev (the OCI default device set, a private devpts, a bounded /dev/shm) where it previously had none.
• kubectl attach for vm Pods. A container with tty: true / stdin: true runs on its own terminal; attach connects to the running process, detaching never kills it, reattaching resumes, and concurrent attaches are allowed. kubectl logs shows the merged stream. Guests advertise their capabilities so an out-of-date guest image reports a clear "recreate the pod" error.
• A node-local image registry. k3sm server --registry-port <port> (off by default; k3sm dev enables it) runs a small OCI registry on loopback. Push locally built images and Pods pull localhost:<port>/name:tag through the ordinary image path. Anonymous pull, credential-protected push, loopback-only bind.
• Images across the cluster. A node advertises its registry to the cluster; a Pod on another node naming a localhost:<port>/… image it lacks falls back to the advertising peers over the encrypted node mesh (content digest-verified on arrival). Linux-guest Pods reach the node registry through the guest-network gateway.
• The image verb set on a stock install. The server serves the runtimed control socket, so k3sm image ls / df / prune / load / import work without a separate daemon — plus new k3sm image pull / tag / untag / inspect / save, and push straight out of the node's store.

Known limitations

• stdinOnce is accepted but behaves as false.
• kubectl attach replays a bounded buffer before following live; redraw a full-screen program with Ctrl-L.
• kubectl attach on the native (non-vm) path remains output-only.

Signed ad-hoc (no Developer ID; not notarized). Full detail in the k3sm CHANGELOG.

v0.1.0

The first release of k3sm — a macOS-native Kubernetes distribution for Apple Silicon. Pods run as native Darwin processes; an experimental vm RuntimeClass boots linux/arm64 images in per-pod micro-VMs against digest-pinned guest artifacts, verified on every boot.

Install (the script verifies checksums and shows what it will do before escalating):

curl -fsSL https://k3sm.io/install.sh | sh

Or pin this build: K3SM_INSTALL_VERSION=0.1.0 curl -fsSL https://k3sm.io/install.sh | sh

The set contains the k3sm binary, the exec shim, both DYLD shims, the entitled k3sm-vmhost helper, and the pre-staged control-plane payload — everything sudo k3sm install requires. Ad-hoc signed, not notarized; the Homebrew tap and notarized package arrive with a later release.

Sources: apis@e90cd77fc1eb · darwin-net@0fd4f3f15131 · runtimed@c52fcab10985 · k3sm@bce0d12