📢 CloudStack Kubernetes Flavors — A Reference Guide I Wish I Had When I Started

[chunkyen]

I've been exploring Kubernetes on CloudStack out of personal interest, and one thing that struck me early on was how scattered the information is. Documentation exists for each individual approach (CKS, CAPC, Talos), but there wasn't a single place that compared them side-by-side or walked through the practical gotchas you hit along the way.

So I started building that reference — a repository covering four approaches to running Kubernetes on CloudStack:

1️⃣ CKS (CloudStack Kubernetes Service) — Native CloudStack integration
2️⃣ CAPC (Cluster API Provider for CloudStack) — Infrastructure-as-Code with CAPI
3️⃣ Rancher + CAPC — Managed Kubernetes with Rancher as the management plane
4️⃣ Talos Linux — Minimal, immutable Kubernetes OS (standalone or with Sidero Omni)

Plus two cross-cutting components that apply to every flavor:

• CloudStack Kubernetes Provider (CCM) — external cloud controller manager for node labels, load balancers, and firewall rules
• CloudStack CSI Driver — persistent storage with dynamic provisioning and snapshots

Each flavor includes architecture analysis, setup guides, and — most importantly — the lessons learned from actually deploying them. Things like the CSI DaemonSet cloud-init-dir issue on Talos, the ClusterClass limitation with CAPC, or the self-signed CA trust dance with Omni. The kind of details that take hours to figure out but are obvious in hindsight.

A significant portion of the work went into air-gapped / offline CKS deployment — including a deep-dive into the ISO build process, container image pre-import workarounds, Cilium manifest patching for offline environments, and the various failure modes you hit when CKS assumes internet access is available. The offline guide (https://github.com/chunkyen/cloudstack-kubernetes-flavors/blob/main/setup/cks/cks-offline.md) documents all of this.

The Rancher + Turtles + CAPC combination was something I figured out on my own — I couldn't find any existing architecture or guide that documented this specific stack working together. It's a good example of how open source technologies can be combined even when there's no official reference architecture, and I hope the guide helps others who want to run this stack without having to piece it together from scratch.

I've also included suggested improvements for the approaches that could benefit from upstream changes:

• CKS Improvements (https://github.com/chunkyen/cloudstack-kubernetes-flavors/blob/main/setup/cks/cks-improvements.md) — pre-import images during upgrades, restrictive default firewall rules, ISO build script fixes, and more
• CAPC + Rancher Turtles Improvements (https://github.com/chunkyen/cloudstack-kubernetes-flavors/blob/main/setup/rancher-turtles-capc/capc-rancher-turtles-improvements.md) — dynamic credential injection, ClusterClass support, etcd backup automation, and other proposals

What's in the repo:

• 🏗 Architecture docs for each flavor and cross-cutting components
• 📖 Step-by-step setup guides
• ⚠️ Known issues and workarounds
• 💡 Suggested improvements for upstream projects
• 📊 Quick comparison table (complexity, GitOps, upgrades, multi-cluster)

How it was built: The entire repository was developed collaboratively with AI agents (Hermes / OpenCLAW) running on locally hosted models (Qwen, Gemma, etc.) — a testament to how far open-source LLMs have come for technical documentation and infrastructure work.

🔗 https://github.com/chunkyen/cloudstack-kubernetes-flavors

This is a lab project driven by curiosity, not production experience. If you're also exploring Kubernetes on CloudStack, I hope this saves you some of the head-scratching. Corrections, contributions, and suggestions are very welcome!

[daviftorres]

Dear @chunkyen,

I am personally interested in what you are doing with Talos inside ACS.

Are you working on alternative offering for Kubernetes using Calico instead of the official K8s?

If that is where you heading, I am available to perform some tests of the Calico build on ACS because that is my preferred way of running K8s anywhere.

Feel free to ping me if you need assistance on that. Regards,

[chunkyen]

Hi, thank you for your interest. May I clarify what do you mean by "Calico instead of the official K8s?". Calico is just a CNI that you installed onto a vanila K8s to provide overlay networking within the cluster. CKS default CNI (what is included in the uploaded CKS iso) is based on Calico.
For me personally, I prefer Cilium as it offers lots of functionlities (like Hubble, Mesh, Gateway API for Ingress), not to mention that is is based on eBPF, so should offer superior performance.
If you are interested in Talos, I have sucessfully demonstrated in the guide that you can deploy on ACS. For CNI, it comes bundle with Flannel. If you want to use Calico, simply disable Fannel as part of the Talos deployment, and install Calico instead. These are all in my guide and you can try those out if that is what you want.

Cheers