Skip to content

Conversation

@aslafy-z
Copy link

@aslafy-z aslafy-z commented Dec 2, 2025

Problem

The Scaleway CCM currently has no way to configure health checks on a port different from the NodePort.

This limitation makes it impossible to properly support Kubernetes services with externalTrafficPolicy: Local.
When you set this policy, Kubernetes:

  1. Allocates a dedicated healthCheckNodePort (e.g., 30100)
  2. Serves pod health status at /healthz on that port
  3. Returns 200 OK only when the node has healthy pods running
  4. Returns 503 Service Unavailable when no healthy pods exist on the node

However, the Scaleway CCM can only health check on the service's NodePort itself, not on the
separate healthCheckNodePort.

Solution

This PR adds the missing capability to configure health checks on a different port by introducing
a new annotation service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service that enables
the Scaleway Load Balancer to use Kubernetes' native healthCheckNodePort.

This enables:

  1. Health checks on a custom port: The load balancer can now health check on healthCheckNodePort
    instead of always using NodePort
  2. Proper externalTrafficPolicy: Local support: Traffic is only sent to nodes with healthy pods
  3. Pod-aware health checking: Uses Kubernetes' built-in /healthz endpoint that knows about pod status
  4. No manual health check configuration needed: Automatic configuration when annotation is set

Closes #192
Replaces #196

Key Features

  • Simple Configuration: Single annotation enables native health checks
  • Flexible Port Selection: Supports boolean values, wildcard, or specific service ports
  • Graceful Fallback: Falls back to NodePort health checks when healthCheckNodePort is unavailable
  • Validation: Comprehensive port range validation (1-65535)
  • Priority: Native health checks take precedence over legacy annotations when both are present

Annotation Syntax

# Enable for all ports
service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service: "true"

# Enable for all ports (wildcard)
service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service: "*"

# Enable for specific service ports
service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service: "80,443"

# Explicitly disable
service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service: "false"

Example Usage

apiVersion: v1
kind: Service
metadata:
  name: my-service
  annotations:
    # NEW: Enable health checks on healthCheckNodePort
    service.beta.kubernetes.io/scw-loadbalancer-health-check-from-service: "true"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local  # Required for healthCheckNodePort allocation 
  # Kubernetes assigns healthCheckNodePort (e.g., 30100) for health checks
  ports:
  - port: 80
    targetPort: 8080
    # Kubernetes assigns NodePort (e.g., 30000) for traffic
  selector:
    app: my-app

Behavior

When annotation is enabled:

Service Port 80 → Load Balancer Backend → Health Check on healthCheckNodePort 30100
                                           ↳ HTTP GET /healthz
                                           ↳ Returns 200 only if healthy pods exist
                                           ↳ Pod-aware health checking
  1. Kubernetes allocates a healthCheckNodePort (e.g., 30100) separate from NodePort
  2. Kubernetes (kube-proxy or cilium) serves health status at http://node-ip:30100/healthz
  3. Scaleway Load Balancer health checks port 30100 (not the NodePort 30000)
  4. Only nodes returning 200 OK receive traffic
  5. Nodes without running pods automatically return 503 from /healthz

When annotation is NOT enabled (default/existing behavior):

Service Port 80 → Load Balancer Backend → Health Check on NodePort 30000
                                           ↳ Checks if port is listening
                                           ↳ NO pod awareness
  1. Health checks use the NodePort as before
  2. No changes to existing functionality
  3. Backward compatibility maintained

Breaking Changes

None. This is a new optional feature that adds a previously unavailable capability. Existing
services continue to work exactly as before.

@damsien
Copy link

damsien commented Dec 3, 2025

Hey @aslafy-z
Just added a test, if it sounds good for you we can squash and then merge!

@aslafy-z
Copy link
Author

aslafy-z commented Dec 3, 2025

Let's go!
Edit: I did not run the tests for your commit

Signed-off-by: Zadkiel AHARONIAN <hello@zadkiel.fr>
@damsien damsien force-pushed the feat/healthchecknodeportannotation branch from 79e836f to 5399f73 Compare December 4, 2025 09:30
@aslafy-z
Copy link
Author

aslafy-z commented Dec 4, 2025

@damsien I upgraded the actions as needed for these checks to start, see #198.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Support dynamic healthCheckNodePort for LoadBalancers

2 participants