-
Notifications
You must be signed in to change notification settings - Fork 139
Description
Summary
Cozystack currently cannot be deployed on non-Talos Kubernetes clusters (kubeadm, k3s, RKE2) because Talos-specific configurations are hardcoded in multiple components.
Problem
Several components assume Talos Linux features:
1. Cilium (packages/system/cilium/values-talos.yaml)
Hardcoded in packages/core/platform/sources/networking.yaml:
valuesFiles:
- values.yaml
- values-talos.yaml # Always appliedContains:
cilium:
k8sServiceHost: localhost
k8sServicePort: 7445 # Talos KubePrism
cgroup:
autoMount:
enabled: false # Talos mounts cgroups2. Cozystack Operator (packages/core/installer/templates/cozystack-operator.yaml)
env:
- name: KUBERNETES_SERVICE_HOST
value: localhost
- name: KUBERNETES_SERVICE_PORT
value: "7445"3. Linstor (packages/system/linstor/templates/satellites-talos.yaml)
LinstorSatelliteConfiguration cozystack-talos removes DRBD module loading init containers (Talos uses system extensions).
4. Kilo (packages/system/kilo/templates/configmap.yaml)
server: https://127.0.0.1:74455. GPU Operator (packages/core/platform/sources/gpu-operator.yaml)
Uses values-talos.yaml with driver.enabled: false.
Why ConfigMap Override Doesn't Work
In v0.30.6, users reported values-cilium in ConfigMap doesn't work on k3s/RKE2 (#973).
Root cause: valuesFiles are merged at artifact build time in packagesource_reconciler.go. Package spec.components[].values CAN override, but bundles/system.yaml only passes KubeOVN values — not Cilium values.
Proposed Solution
Option A: Add networking.apiServer to Platform values (Recommended)
Add to packages/core/platform/values.yaml:
networking:
apiServer:
host: "localhost" # default for Talos KubePrism
port: "7445"Template in bundles/system.yaml:
{{- $ciliumValues := dict "cilium" (dict
"k8sServiceHost" .Values.networking.apiServer.host
"k8sServicePort" .Values.networking.apiServer.port) -}}
{{- $_ := set $networkingComponents "cilium" (dict "values" $ciliumValues) -}}Non-Talos users override:
networking:
apiServer:
host: "10.0.0.1"
port: "6443"Option B: Add generic variants
Add cilium-generic, gpu-operator-generic variants without values-talos.yaml.
Additional Changes
- Make
KUBERNETES_SERVICE_HOST/PORTin cozystack-operator.yaml configurable - Conditional
LinstorSatelliteConfiguration cozystack-taloscreation - Template Kilo kubeconfig server URL
Use Cases
- Running Cozystack on existing kubeadm clusters
- Running Cozystack on k3s/RKE2 where Talos is not an option
- Testing Cozystack in kind/minikube for development
Related
- Example: deploy Cozystack on top of plain Ubuntu using kubeadm #973: Community kubeadm deployment example (v0.30.6)