Skip to content

docs: explain proxyrunner session-aware backend pod routing when using backendReplicas #708

@yrobla

Description

@yrobla

Problem

guides-k8s/run-mcp-k8s.mdx documents the backendReplicas field and states that Redis is required when replicas > 1, but it does not explain why Redis is needed or how the proxyrunner routes sessions to specific backend pods.

When backendReplicas > 1, the proxyrunner uses a Redis-backed RoutingStorage to pin each session to the specific backend pod that initialized it. Without this explanation, users may:

  • Not understand why Redis is needed just for backend scaling (not proxy scaling)
  • Misconfigure or skip Redis for backendReplicas scenarios
  • Not understand what happens when a backend pod is restarted or replaced

What's needed

Add a short explanation (a paragraph or callout) in the run-mcp-k8s.mdx#horizontal-scaling section that covers:

  1. Why Redis is needed when scaling backends: the proxyrunner must route all requests within a session to the backend pod that initialized that session (stateful MCP connections). Redis stores the session-to-backend-pod mapping so all proxyrunner replicas share this state.
  2. What happens on backend pod restart: the session-to-pod mapping in Redis is invalidated; the next request reconnects to an available pod (sessions are not automatically migrated).
  3. The note about SessionStorageWarning: clarify that the condition only fires for replicas > 1 (proxy scaling), not for backendReplicas > 1 — but Redis is still recommended for backendReplicas > 1 since without it, session routing relies solely on client-IP affinity which is unreliable behind NAT (this is already documented but the implication for backendReplicas should be made explicit).

References

  • guides-k8s/run-mcp-k8s.mdx#horizontal-scaling — section to update
  • Epic: stacklok/stacklok-epics#263 (Horizontal Scaling for Proxyrunner, THV-0047)
  • RFC THV-0047 — describes the RoutingStorage design

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions