Session Affinity or Sticky Session

[harrycch]

Does the New API supports a similar feature to Session Affinity of LiteLLM? (https://docs.litellm.ai/docs/proxy/auto_routing#session-affinity)

[AbdullahTammar]

Yes—the current main branch has channel affinity, which provides this kind of sticky routing. A rule extracts a stable key from a request header or JSON-body path, matches the request/model, and caches the selected channel with a TTL.

For a custom application, configure a rule with a request_header key source (for example, X-Session-ID) and send the same opaque session ID on each request in that conversation. The rule also supports group/model scoping and a session mode. Keep the ID stable across turns; generating a fresh value per request defeats affinity.

See the settings definition and channel-affinity implementation. This is channel selection, not storage of conversation history or a guarantee that the upstream provider uses the same internal worker.

This confirms the feature in current source, not its availability in every released image. Which New API version/tag are you running? That determines which settings and modes are available.