[Release-1.27] - "NodePasswordValidationFailed" Event is created on server restart when agent is disabled

[brandond]

Backport fix for "NodePasswordValidationFailed" Event is created on server restart when agent is disabled

• "NodePasswordValidationFailed" Event is created on server restart when agent is disabled #9447

[aganesh-suse]

Validated on release-1.27 branch with commit 11b31c2

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Cluster Configuration:

HA: 3 server/ 1 agent

Config.yaml:

token: xxxx
cluster-init: true
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
node-label:
- k3s-upgrade=server

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s

curl -sfL https://get.k3s.io | sudo INSTALL_K3S_COMMIT='11b31c2815c83e424b9888b71e2826e73ca8403c' sh -s - server

3. Edit the /etc/rancher/k3s/config.yaml file of server nodes to have extra line:

disable-agent: true

Restart the k3s service: sudo systemctl restart k3s
4. Search for 'NodePasswordValidationFailed' event from journal logs of all server nodes

journalctl -xeu k3s | grep 'Password'

Replication Results:

• k3s version used for replication:

$ k3s -v
k3s version v1.27.10+k3s2 (5b2ac881)
go version go1.20.13

journalctl -xeu k3s | grep Password
Feb 16 23:44:18 ip-172-31-19-163 k3s[63970]: I0216 23:44:18.887429   63970 event.go:307] "Event occurred" object="ip-172-31-19-163" fieldPath="" kind="Node" apiVersion="" type="Normal" reason="NodePasswordValidationComplete" message="Deferred node password secret validation complete"
Feb 16 23:48:49 ip-172-31-19-163 k3s[64104]: time="2024-02-16T23:48:49Z" level=info msg="Password verified locally for node ip-172-31-19-163"
Feb 16 23:49:00 ip-172-31-19-163 k3s[64104]: I0216 23:49:00.095879   64104 event.go:307] "Event occurred" object="ip-172-31-19-163" fieldPath="" kind="Node" apiVersion="" type="Normal" reason="NodePasswordValidationComplete" message="Deferred node password secret validation complete"

Validation Results:

• k3s version used for validation:

$ k3s -v
k3s version v1.27.11-rc1+k3s1 (11b31c28)
go version go1.21.7

journalctl -xeu k3s | grep Password