Skip to content

feat(platform): support non-Talos Kubernetes deployments #1933

@lexfrei

Description

@lexfrei

Summary

Cozystack currently cannot be deployed on non-Talos Kubernetes clusters (kubeadm, k3s, RKE2) because Talos-specific configurations are hardcoded in multiple components.

Problem

Several components assume Talos Linux features:

1. Cilium (packages/system/cilium/values-talos.yaml)

Hardcoded in packages/core/platform/sources/networking.yaml:

valuesFiles:
- values.yaml
- values-talos.yaml   # Always applied

Contains:

cilium:
  k8sServiceHost: localhost
  k8sServicePort: 7445   # Talos KubePrism
  cgroup:
    autoMount:
      enabled: false     # Talos mounts cgroups

2. Cozystack Operator (packages/core/installer/templates/cozystack-operator.yaml)

env:
- name: KUBERNETES_SERVICE_HOST
  value: localhost
- name: KUBERNETES_SERVICE_PORT
  value: "7445"

3. Linstor (packages/system/linstor/templates/satellites-talos.yaml)

LinstorSatelliteConfiguration cozystack-talos removes DRBD module loading init containers (Talos uses system extensions).

4. Kilo (packages/system/kilo/templates/configmap.yaml)

server: https://127.0.0.1:7445

5. GPU Operator (packages/core/platform/sources/gpu-operator.yaml)

Uses values-talos.yaml with driver.enabled: false.

Why ConfigMap Override Doesn't Work

In v0.30.6, users reported values-cilium in ConfigMap doesn't work on k3s/RKE2 (#973).

Root cause: valuesFiles are merged at artifact build time in packagesource_reconciler.go. Package spec.components[].values CAN override, but bundles/system.yaml only passes KubeOVN values — not Cilium values.

Proposed Solution

Option A: Add networking.apiServer to Platform values (Recommended)

Add to packages/core/platform/values.yaml:

networking:
  apiServer:
    host: "localhost"    # default for Talos KubePrism
    port: "7445"

Template in bundles/system.yaml:

{{- $ciliumValues := dict "cilium" (dict
  "k8sServiceHost" .Values.networking.apiServer.host
  "k8sServicePort" .Values.networking.apiServer.port) -}}
{{- $_ := set $networkingComponents "cilium" (dict "values" $ciliumValues) -}}

Non-Talos users override:

networking:
  apiServer:
    host: "10.0.0.1"
    port: "6443"

Option B: Add generic variants

Add cilium-generic, gpu-operator-generic variants without values-talos.yaml.

Additional Changes

  • Make KUBERNETES_SERVICE_HOST/PORT in cozystack-operator.yaml configurable
  • Conditional LinstorSatelliteConfiguration cozystack-talos creation
  • Template Kilo kubeconfig server URL

Use Cases

  • Running Cozystack on existing kubeadm clusters
  • Running Cozystack on k3s/RKE2 where Talos is not an option
  • Testing Cozystack in kind/minikube for development

Related

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions