Skip to content

0.11.4

Choose a tag to compare

@github-actions github-actions released this 11 Nov 00:09
· 23 commits to main since this release
0e7d06f

Two fixes from real questions. With SGLang's --enable-session-radix-cache, a client's own body
session_id now reaches the engine, and a session sent only as an x-session-id header is named when
its prefix is evicted. A tool whose schema lists required in another order is now the same tool, for
both the diagnostic and --fix canonical_tools.

  • SGLang with --enable-session-radix-cache: a client's body session_id reaches the engine, and a dropped session header is named. That flag makes SGLang evict KV that no session holds first, and SGLang reads the session from the body session_id only. Prefixwise removed a client's body session_id on its way to SGLang unless its own KV-event join was on, which it never is with this flag, so the client's sessions lost their protection; with the flag on (read from /server_info) it now passes it through. Without the flag nothing changes: there it makes no difference to the cache. A client that names its session only in an x-session-id header (which SGLang does not read, and which per sglang#47763 sgl-router does not forward) gets a one-time warning, and an eviction of its prefix gets the detail session_header_dropped, with the fix: send the id as a body session_id too (SGLang PR #45620 would read the header; not merged). It is a detail of evicted, not a cause of its own: the miss is an eviction, and measured live it happens only with that flag on and memory full: 4,000 of 4,012 tokens cached with a body session_id, 3 without; with the flag off, 3 either way (record). Turn facts gain session_unprotected, sent only when true: a 0.11.3 prefixwise collect refuses those records as malformed, so upgrade the collector first if you run SGLang with that flag.
  • A tool whose schema lists required in another order is the same tool. JSON Schema's required (and a list type, such as ["string", "null"]) is a set, and a schema built from a Python set lists it in a different order after a restart. Prefixwise called that tool_catalog_change, a legitimate change, and --fix canonical_tools sent it as it came, so the miss was neither counted as waste nor fixed. Now the detector names it tool_reserialize and the fix sorts those two arrays, at any depth; enum, anyOf and every other array keep their order, and default, example and const values are never touched. Key order was already sorted at every depth. From a Reddit question (r/LLMDevs) about nested tool schemas. Such a tool hashes differently from before, so across the upgrade (a restart, or an edge and a collector on different versions) expect one tool change per conversation that uses one; upgrade an edge and its collector together.