Kama (Kubernetes Llama) is being built as a Kubernetes-native control and data plane
for running GGUF models with llama.cpp and llama-server. It will import and
retain models, determine a feasible CPU/GPU topology, create serving workloads,
route requests across ready replicas, and scale model capacity to and from zero.
The M0 engineering foundation and M1 persistent artifact plane are complete. M1 contains the v1alpha1 artifact API, importer and persistent-cache implementation, packaging, examples, and recovery guidance; CI, the Kubernetes 1.34-1.36 matrix, public Hugging Face E2E, and storage resilience E2E are verified. The project owner closed M1 with the protected private-source repetition recorded as a non-blocking follow-up, not as evidence that already passed.
The M2 baseline-serving repository implementation is also present: ModelDeployment
provides explicit CPU or one-NVIDIA-GPU intent, the controller creates one internal
serving workload from a ready artifact, and Kama-owned CPU/CUDA llama.cpp runtimes
share a one-shot supervisor with readiness, diagnostics, and drain behavior. M2
remains In Progress until its hosted CPU/failure and protected real-NVIDIA acceptance
evidence is recorded.
The canonical, modular plan starts at docs/project-plan/README.md.
The current v1 direction is:
- Go and
controller-runtime, distributed as an Apache-2.0 open-source operator. - Namespaced CRDs and
kubectlas the primary management interface. - NVIDIA-first GPU support through the device plugin and GPU Feature Discovery, with an internal accelerator abstraction for future backends.
- Independent model replicas across Pods and same-node multi-GPU inference within one Pod; cross-node llama.cpp RPC is outside the production v1 boundary.
- Persistent, verified GGUF artifacts from Hugging Face or manually populated PVCs.
- A llama-server-compatible gateway with capacity-aware routing and KEDA-backed scale-to-zero.
M0 and M1 are complete; M2 is in progress. M0 completion is backed by the passing CI run, Kubernetes 1.34-1.36 compatibility matrix, and verified commit. Accepted decisions, assumed defaults, and remaining validation items are tracked in decisions and open questions.
Start with the artifact-plane examples, consult the
artifact recovery and
model serving runbooks. The unexecuted
private-source M1 repetition remains transparently tracked in the
M1 acceptance record without reopening the closed milestone.
The reusable real-cluster suites and extension contract live under
test/e2e. The Helm chart is the complete Kama installation path,
including the artifact plane and baseline-serving resources. config/default remains
a developer Kustomize bundle with
admission disabled until a downstream overlay provides webhook TLS.