Add sticky session routing to pin conversations to accounts#2
Merged
Conversation
Each conversation is now pinned to the account that handled its first
request, identified by `prompt_cache_key` (forwarded to the backend as
`session_id`/`conversation_id`). Subsequent turns in the same session
reuse the same account so the backend's KV cache and reasoning state
remain valid. New sessions are still distributed across accounts using
the configured rotation strategy, so multi-account load-balancing is
preserved at the session level rather than the request level.
New settings in RotationSettings (all with safe defaults):
stickySessionRouting – enable/disable (default: true)
sessionIdleTimeoutMs – evict after idle (default: 1 h)
sessionStickyFallback – 'rotate' | 'fail' when pinned account is
unavailable (default: 'rotate')
New file src/session-store.ts: lightweight in-memory map with TTL
pruning, alias-level bulk eviction, and a dashboard-ready list view.
New dashboard API endpoints:
GET /api/sessions list active session mappings
DELETE /api/sessions/:id evict one session
DELETE /api/accounts/:alias/sessions evict all sessions for alias
GET /api/state now includes sessions.count and sessions.byAlias.
https://claude.ai/code/session_0128Ftvfrm7VHDzxhj29nv1p
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements sticky session routing to pin each conversation to the same account across multiple turns, preserving the backend's KV cache and reasoning state. Sessions are identified by
prompt_cache_keyand mapped to accounts in memory with automatic idle-based eviction.Key Changes
New
session-store.tsmodule: In-memory session→account mapping with:getSessionAlias()/setSessionAlias()for session pinningclearSession()/clearSessionsForAlias()for cleanuplistSessions(),sessionCount(),sessionCountByAlias()Updated
rotation.ts:sessionIdfield toAccountSelectionContextinterfacegetNextAccount()to reuse pinned account if healthyOPENCODE_MULTI_AUTH_DEBUGenvironment variableNew settings in
types.ts:stickySessionRouting(boolean, default: true)sessionIdleTimeoutMs(number, default: 3,600,000 ms / 1 hour)sessionStickyFallback('rotate' | 'fail', default: 'rotate')Web console API endpoints in
web.ts:GET /api/sessions– list all active sessionsDELETE /api/sessions/{sessionId}– clear a specific sessionDELETE /api/accounts/{alias}/sessions– clear all sessions for an accountImplementation Details
unref()to avoid keeping the process alivehttps://claude.ai/code/session_0128Ftvfrm7VHDzxhj29nv1p