Repository navigation
v0.7.153
Cache-placed sessions stay on their rung: when a host's cache placement names a rung that authors a concurrency bound with saturation=refuse, a per-worker shed there (queue_bound, fair_share_shed, rate window) no longer spills the conversation to a sibling rung holding none of its prompt cache. Free callers get the lane_saturated 429 (Retry-After 5) and retry onto the same warm rung; priority callers overflow that rung up to their ceiling. Real failures still fail over; requests without a placement balance as before. New GatewayRoute.cache_placed_deployment_id. Native unchanged at 0.3.130.