generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 618
Closed
Description
Currently the echo server provides pod name, namespace, etc in the response headers. For use cases such as running multiple echo servers in a single pod, the echo server should optionally support echo'ing the HTTP server listening port number in the response headers. For example:
# Data Parallelism (DP) backend deployment: 3 pods, each listening on three ports to simulate vLLM ranks
apiVersion: apps/v1
kind: Deployment
metadata:
name: dp-inference-model-server-deployment
namespace: inference-conformance-app-backend
labels:
app: dp-inference-model-server
spec:
replicas: 3
selector:
matchLabels:
app: dp-inference-model-server
template:
metadata:
labels:
app: dp-inference-model-server
spec:
containers:
- name: echoserver-3000
image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
ports:
- containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2
env:
- name: HTTP_PORT # Default port for HTTP echo server
value: "3000"
- name: H2C_PORT # Default port for HTC echo server
value: "3001"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: echoserver-3002
image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
ports:
- containerPort: 3002
readinessProbe:
httpGet:
path: /
port: 3002
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2
env:
- name: HTTP_PORT
value: "3002"
- name: H2C_PORT
value: "3003"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: echoserver-3004
image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
ports:
- containerPort: 3004
readinessProbe:
httpGet:
path: /
port: 3004
initialDelaySeconds: 3
periodSeconds: 5
failureThreshold: 2
env:
- name: HTTP_PORT
value: "3004"
- name: H2C_PORT
value: "3005"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIPxref GIE issue: kubernetes-sigs/gateway-api-inference-extension#1680
robscott
Metadata
Metadata
Assignees
Labels
No labels