Repository navigation
0.11.4
Two fixes from real questions. With SGLang's --enable-session-radix-cache, a client's own body
session_id now reaches the engine, and a session sent only as an x-session-id header is named when
its prefix is evicted. A tool whose schema lists required in another order is now the same tool, for
both the diagnostic and --fix canonical_tools.
- SGLang with
--enable-session-radix-cache: a client's bodysession_idreaches the engine, and a dropped session header is named. That flag makes SGLang evict KV that no session holds first, and SGLang reads the session from the bodysession_idonly. Prefixwise removed a client's bodysession_idon its way to SGLang unless its own KV-event join was on, which it never is with this flag, so the client's sessions lost their protection; with the flag on (read from/server_info) it now passes it through. Without the flag nothing changes: there it makes no difference to the cache. A client that names its session only in anx-session-idheader (which SGLang does not read, and which per sglang#47763 sgl-router does not forward) gets a one-time warning, and an eviction of its prefix gets the detailsession_header_dropped, with the fix: send the id as a bodysession_idtoo (SGLang PR #45620 would read the header; not merged). It is a detail ofevicted, not a cause of its own: the miss is an eviction, and measured live it happens only with that flag on and memory full: 4,000 of 4,012 tokens cached with a bodysession_id, 3 without; with the flag off, 3 either way (record). Turn facts gainsession_unprotected, sent only when true: a 0.11.3prefixwise collectrefuses those records as malformed, so upgrade the collector first if you run SGLang with that flag. - A tool whose schema lists
requiredin another order is the same tool. JSON Schema'srequired(and a listtype, such as["string", "null"]) is a set, and a schema built from a Python set lists it in a different order after a restart. Prefixwise called thattool_catalog_change, a legitimate change, and--fix canonical_toolssent it as it came, so the miss was neither counted as waste nor fixed. Now the detector names ittool_reserializeand the fix sorts those two arrays, at any depth;enum,anyOfand every other array keep their order, and default, example and const values are never touched. Key order was already sorted at every depth. From a Reddit question (r/LLMDevs) about nested tool schemas. Such a tool hashes differently from before, so across the upgrade (a restart, or an edge and a collector on different versions) expect one tool change per conversation that uses one; upgrade an edge and its collector together.