RKE2 Provider with DRBD best practices

[philippl-onz]

Hi everyone,

I am working on a project based on RKE2 and stumbled over Kairos. I really like the concept but i am a bit confused about the RKE2 Provider (https://github.com/kairos-io/provider-rke2) which seems to be kind of official but i couldn't really find any useful documentation about it.

so i basically have three main questions:

1. how to utilize the official rke2 provider? i tried

FROM quay.io/kairos/kairos-init:latest AS kairos-init
FROM ubuntu:24.04

ARG RKE2_VERSION=v1.32.9+rke2r1
ARG VERSION=1.0.0-rke2-cilium-drbd9

ENV DEBIAN_FRONTEND=noninteractive
RUN --mount=type=bind,from=kairos-init,src=/kairos-init,dst=/kairos-init \
    /kairos-init \
      --version "${VERSION}" \
      --model generic \
      --provider rke2 \
      --provider-rke2-version "${RKE2_VERSION}"

but that seems to do nothing and i dont get any rke2 binaries in my final image.

to workaround this image we just installed RKE2 the normal way after that and copied the binaries into /usr/bin/ to get persistence but that seems to be a bit of an ugly solution in my books.

2. what would be the correct way of defining network stuff in the cloud config? i tried the native cloud-config network: block but it doesn't seem to apply here.

3. because we need DRBD we are installing it also in the Dockerfile which works perfectly finde but i am also not 100% sure if this is the way it is meant to be:

FROM quay.io/kairos/kairos-init:latest AS kairos-init
FROM ubuntu:24.04
...
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get install -y \
      software-properties-common \
      ca-certificates \
      curl \
      gnupg \
      dkms \
      build-essential \
      linux-generic-hwe-24.04 \
      linux-headers-generic-hwe-24.04 \
      drbd-utils \
      lvm2 \
      thin-provisioning-tools \
      dmsetup \
      nvme-cli \
      qemu-guest-agent \
      open-iscsi \
      nfs-common \
      jq && \
    add-apt-repository -y ppa:linbit/linbit-drbd9-stack && \
    apt-get update && \
    apt-get install -y drbd-dkms drbd-utils && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

If someone could point me into the right direction of how a setup based on RKE2 with DRBD with fixed ip's should look like, i would highly appreciate it.

thanks

[wcrum]

Thanks for the question @philippl-onz!

So by default, kairos-init supports provider-kairos, where a "provider" is like a plugin that extends the functions of Kairos.

https://kairos.io/docs/reference/configuration/#provider-configs

Given that by default kairos-init does not have the provider-rke2 you will need to include that provider in the Dockerfile.

https://kairos.io/docs/architecture/providers/

Step 1 - Build Provider build the provider binary.

This binary seems to be built via Earthly, so running earthly --platform=linux/amd64 +build-provider will output a build/agent-provider-rke2 binary which you can copy into your Kairos Dockerfile.

Step 2 - Kairos Init - ensure that is available at /system/providers/agent-provider-rke2

COPY ./build/agent-provider-rke2 /system/providers/agent-provider-rke2

It seems like part of the installation will require the install_rke2.sh which is in charge of downloading necessary components. That will go out and download rke2 from releases and extract and install it onto the Image, then enabling relevant Systemd services.

ENV INSTALL_RKE2_VERSION=v1.30.0
COPY install_rke2.sh /tmp/install_rke2.sh
RUN /tmp/install_rke2.sh && rm /tmp/install_rke2.sh

Continue with kairos-init

RUN --mount=type=bind,from=kairos-init,src=/kairos-init,dst=/kairos-init \
  /kairos-init --version "${VERSION}"

Step 3 - Cloud Config - ensure that your Cloud Config matches the provider-rke2 required schema.

The provider-rke2 is a plugin, uses the kairos-sdk Cluster Plugin.

https://github.com/kairos-io/kairos-sdk/blob/28594ae48f866c3919d51ff62a4c9f7e6a920deb/clusterplugin/config.go#L41-L72

#cloud-config

hostname: rke2-cp-1

users:
- name: kairos
  passwd: kairos
  groups:
  - admin

cluster:
  cluster_token: "super-secret-shared-token"
  role: init
  config: |
    node-name: rke2-cp-1
    cluster-cidr: "10.42.0.0/16"
    service-cidr: "10.43.0.0/16"
    tls-san:
    - 10.0.0.10
    - rke2.example.com
    disable:
    - rke2-ingress-nginx

Please review - https://docs.rke2.io/reference/server_config.

[wcrum]

Checking in @philippl-onz, let me know if this resolves your issue.