[Release-1.26] - Split server setup fails on etcd-only, cp-only config

[brandond]

Backport fix for Split server setup fails on etcd-only, cp-only config

• Split server setup fails on etcd-only, cp-only config #8672

[ShylajaDevadiga]

Validated using k3s version v1.26.10-rc1+k3s1 (10379aa)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
Ubuntu 22.04

Cluster Configuration:
3 server 1 agent

Config.yaml:
etcd-only

$ cat config.yaml 
token: secret
disable-apiserver: true
disable-scheduler: true
disable-controller-manager: true
node-taint: "node-role.kubernetes.io/etcd=true:NoExecute"
node-external-ip: <ip>

cp-only

$ cat config.yaml 
token: secret
server: https://serverIP:6443
disable-etcd: true
node-taint: "node-role.kubernetes.io/control-plane=true:NoSchedule"

Steps to reproduce the issue and validate the fix

1. Copy config.yaml
2. Install k3s

Validation results:
Validated nodes join correctly and basic functionality works

ubuntu@ip-172-31-6-232:~$ k3s -v
k3s version v1.26.10-rc1+k3s1 (10379aa3)
go version go1.20.10
ubuntu@ip-172-31-6-232:~$ kubectl get nodes
NAME               STATUS   ROLES                  AGE    VERSION
ip-172-31-10-184   Ready    control-plane,master   118s   v1.26.10-rc1+k3s1
ip-172-31-10-254   Ready    control-plane,master   96s    v1.26.10-rc1+k3s1
ip-172-31-5-50     Ready    <none>                 83s    v1.26.10-rc1+k3s1
ip-172-31-6-232    Ready    etcd                   89s    v1.26.10-rc1+k3s1
ubuntu@ip-172-31-6-232:~$ kubectl get pods -A
NAMESPACE     NAME                                      READY   STATUS      RESTARTS   AGE
kube-system   coredns-59b4f5bbd5-fcj7g                  1/1     Running     0          66s
kube-system   helm-install-traefik-crd-w6bqk            0/1     Completed   0          67s
kube-system   helm-install-traefik-tsmfr                0/1     Completed   1          67s
kube-system   local-path-provisioner-76d776f6f9-2gmpv   1/1     Running     0          66s
kube-system   metrics-server-68cf49699b-cslt9           1/1     Running     0          66s
kube-system   svclb-traefik-2015d60e-4sls9              2/2     Running     0          55s
kube-system   svclb-traefik-2015d60e-jws7b              2/2     Running     0          55s
kube-system   svclb-traefik-2015d60e-prkjj              2/2     Running     0          55s
kube-system   traefik-f75d47c4b-8wq7x                   1/1     Running     0          55s