Feature Discussion k8s plugin

[jshi991]

Overview

The k8s plugin adds Kubernetes development cluster support to the container CLI. Cluster nodes run k3s, a lightweight Kubernetes distribution, packaged as a container image that runs directly on the Apple container runtime.

This is an experimental feature. The goal is to make spinning up a local Kubernetes cluster fast and simple.

Proposed UX

Intial feature set - run , create, delete, loadImage, write-config, list

The initial feature set covers the core day-to-day workflow: boot a single-node cluster, load images into it, point kubectl at it, and tear it down when finished.

k3s base image — Before any cluster can start the plugin needs a k3s node image. run and create check for the image on first use and build it automatically if it is missing. The image layers rancher/k3s with a custom entrypoint that handles Apple-kernel specifics: native containerd snapshotter, cgroup v2 setup, and the sysctl values required for Kubernetes networking.

run / create / delete — run creates and starts a single control-plane cluster in one step and writes a kubeconfig entry to ~/.kube/config automatically. create creates the container without starting it. delete stops and removes the cluster and cleans up its entries from ~/.kube/config.

container k8s run my-cluster
container k8s create my-cluster
container k8s delete my-cluster
container k8s delete --force my-cluster

load-image — Loads a locally-stored image into the cluster's containerd so kubelet can schedule pods using it without an external registry. The image is saved from the container image store to a temporary OCI tar and streamed into the node via ctr images import. Short references (alpine) are automatically qualified to docker.io/library/alpine:latest.

container k8s load-image my-cluster my-app:latest

write-config / get-kubeconfig — write-config merges the cluster's kubeconfig into ~/.kube/config and sets current-context. get-kubeconfig prints the YAML to stdout for scripts or manual merging. Both rewrite the API server address to 127.0.0.1:<host-port> or to the machine's FQDN when one is detected.

container k8s write-config my-cluster
container k8s get-kubeconfig my-cluster

list — Lists all clusters and their nodes in a table, discovered via the plugin=k8s container label.

container k8s list
# CLUSTER     NODE        ROLE           STATE    CPUS  MEMORY   ADDR          PORTS
# my-cluster  my-cluster  control-plane  running  4     4096 MB  192.168.64.5  6445->6443

Proposed Architecture

Future Work

API Support - all cluster topology requests will map to a well defined configuration object

Multi-node clusters — run will accept a --workers N flag to start N worker nodes alongside the control plane. Each worker runs k3s in agent mode and joins using a token fetched from the control plane. load-image will target all nodes by default so every kubelet can schedule pods regardless of placement.

Load balancing — Allow clusters to expose services through a host-accessible endpoint without manual port-forwarding. k3s ships with ServiceLB; the integration point will be designed to map cleanly to the container networking model.

Registries — Let users point a cluster at a local registry (e.g. one started with container run) so that images pushed there are immediately pullable inside the cluster without running load-image manually. This maps to the k3s registries.yaml mirror configuration.

Questions for the Community

We'd love your feedback:

1. Is this something you'd use? Would this be valuable to your workflow?
2. What features are must-haves?
3. What use cases would this unlock for you?

---

Please share your thoughts and feedback/suggestions. We want to hear from you!

[afbjorklund]

Shouldn't the plugin rather be called k3s? It is expected to run kubeadm for k8s