Skip to content

Fix API load balancer failover health check#46

Merged
FriedCircuits merged 1 commit into
mainfrom
bugfix/k8s-api-lb-keepalived-health-check
Jul 9, 2026
Merged

Fix API load balancer failover health check#46
FriedCircuits merged 1 commit into
mainfrom
bugfix/k8s-api-lb-keepalived-health-check

Conversation

@FriedCircuits

@FriedCircuits FriedCircuits commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the Kubernetes API load balancer failover check so keepalived tracks whether the local HAProxy frontend can actually reach the API, not just whether the HAProxy process is running.

Background

During a control-plane shutdown, one LB container kept the VIP even though its HAProxy frontend was accepting TCP connections and then closing during TLS. The other LB containers and the remaining control-plane API servers were healthy, but clients using the VIP could not connect because keepalived only checked systemctl is-active haproxy.

Changes

  • Install curl in the LB containers.
  • Render /usr/local/sbin/k8s-api-lb-health-check.sh during bootstrap.
  • Check https://127.0.0.1:<frontend-port>/readyz through the local HAProxy frontend.
  • Update keepalived to track the local readiness script instead of only HAProxy service state.
  • Document the new readiness behavior.

Validation

  • terraform fmt -check
  • terraform init -backend=false -input=false
  • terraform validate
  • git diff --check -- modules/proxmox/k8s-api-lb
  • Deployed to the cluster and repeated the control-plane shutdown test. The VIP stayed usable while the backend dropped out, and etcd quorum remained healthy on the surviving control-plane nodes.

@FriedCircuits
FriedCircuits marked this pull request as ready for review July 9, 2026 04:33
@FriedCircuits
FriedCircuits merged commit 1e3a4f3 into main Jul 9, 2026
@FriedCircuits
FriedCircuits deleted the bugfix/k8s-api-lb-keepalived-health-check branch July 9, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant