[Release-1.26] - CA cert validation failure on additional server nodes prevents joining without explicitly configuring node-external-ip

[brandond]

Backport fix for CA cert validation failure on additional server nodes prevents joining without explicitly configuring node-external-ip

• CA cert validation failure on additional server nodes prevents joining without explicitly configuring node-external-ip #8247

[VestigeJ]

##Environment Details
VERSION=v1.26.8-rc1+k3s1
VERSION=v1.26.8-rc2+k3s1

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

ami-0bbc06589f2e4f4f2

Linux 5.14.21-150500.53-default x86_64 GNU/Linux

PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME                    STATUS   ROLES                       AGE   VERSION
node/ip-31-31-31-121    Ready    control-plane,etcd,master   11s   v1.26.8-rc2+k3s1
node/ip-31-31-31-47     Ready    control-plane,etcd,master   58s   v1.26.8-rc2+k3s1

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
protect-kernel-defaults: true
cluster-init: true
tls-san-security: true
node-external-ip: 3.3.6.9

server: https://3.3.6.9:6443
write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
protect-kernel-defaults: true
tls-san-security: true

Reproduction

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf
$ sudo cp 90-kubelet.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_K3S_VERSION=v1.26.8-rc1+k3s1 INSTALL_K3S_EXEC=server ./install-k3s.sh
$ // on second node join primary node server - observe the failure

Results:

CA cert validation failed

Validation

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf
$ sudo cp 90-kubelet.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_K3S_VERSION=v1.26.8-rc2+k3s1 INSTALL_K3S_EXEC=server ./install-k3s.sh
$ // on second node join primary node server - observe the cluster starts as expected

Results:

NAME                    STATUS   ROLES                       AGE   VERSION
node/ip-31-31-31-121    Ready    control-plane,etcd,master   11s   v1.26.8-rc2+k3s1
node/ip-31-31-31-47     Ready    control-plane,etcd,master   58s   v1.26.8-rc2+k3s1

Primary server omitted secondary server omitted ✅
Primary server true secondary server true ✅
Primary server false secondary server true cluster still joins with errors ✅
Primary server true secondary server false cluster fails ❌