Ext-Proc Cluster Health Checking

[danehans]

Currently, the Envoy static config used for e2e testing does not implement health checking. Envoy and EPP are configured for Kubelet health checking:

Envoy logs:

[2025-01-28 22:03:20.933][24][debug][http] [source/common/http/conn_manager_impl.cc:1183] [Tags: "ConnectionId":"168","StreamId":"3271118745624594159"] request headers complete (end_stream=true):
':authority', '10.244.0.174:19001'
':path', '/ready'
':method', 'GET'
'user-agent', 'kube-probe/1.27'
'accept', '*/*'
'connection', 'close'

EPP logs:

I0128 21:58:29.531032       1 health.go:22] gRPC health check serving: service:"inference-extension"

However, no active health checking is configured for the ext_proc cluster. Should this be added?

xref

[danehans]

cc: @ahg-g

[ahg-g]

How would that look like?

[danehans]

gRPC health_checks for the ext-proc cluster. Something like this:

clusters:
  - name: ext_proc
    ...
    health_checks:
      - timeout: 2s                # How long Envoy waits for a health-check response
        interval: 5s               # How often Envoy sends health checks
        unhealthy_threshold: 2     # Number of consecutive failures to mark EPP unhealthy
        healthy_threshold: 2       # Number of consecutive successes to mark EPP healthy
        grpc_health_check:
          # See https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/manifests/ext_proc.yaml#L91-L93
          service_name: "inference-extension" 
          authority: "$INFER_EXT_SVC_NAME.$INFER_EXT_SVC_NS:$INFER_EXT_HEALTH_PORT"
...

If ^ works as expected using the static Envoy config, consider including in the ext-proc protocol spec.

[ahg-g]

Sounds good to me.

[zhengkezhou1]

let me try this

/assign