0.12.1

For anyone comparing two ways of running the same agent work: prefixwise report --compare TASK TASK puts two tasks side by side with the difference, so one long session against several short ones (or one prompt layout against another) is one command, not a spreadsheet. SGLang and LMCache are now named in the package's one-line description and keywords on PyPI.

• Two tasks side by side: prefixwise report --compare TASK TASK. The same work done two ways (one long session against several short ones, one prompt layout against another) as two columns and the change between them: requests, prompt tokens, the share from cache, output tokens, avoidable tokens, cost and outcome. Asked for by someone comparing long and short sessions on their own gateway: the report summed the two tasks together, and the per-task table (--by task_id) was there but not mentioned. The report now points to both when a ledger holds more than one task. Cost is shown only for a task whose every request is priced, with enough digits to compare small runs.
• SGLang and LMCache in the PyPI summary and keywords. Both were supported and named in the README already; the one-line description and keywords now name them too.

0.12.0

For anyone deciding whether to trust the causes Prefixwise names: the decision procedure is now written down (docs/ALGORITHM.md), checked by invariants on generated traffic, audited twice against the code, and measured at scale on two engines. Through vLLM and SGLang, half of it replaying real Claude Code trajectories, every one of 1,772 injected changes was seen, none was named as another cause, and none of 3,622 turns that only appended got a change cause. Also new: prefixwise import gateway replays a gateway's request log through the proxy's own pipeline, so Prefixwise can be tried on yesterday's traffic without touching production; the report says how far fixing the avoidable misses would take the hit rate; prefixwise.wrap records streamed calls; and OpenRouter keeps its sticky routing through Prefixwise. 0.11.8 was prepared but never published: its changes are here.

• The algorithm, written down and checked. docs/ALGORITHM.md specifies the decision procedure: what is compared, the order of the rules, the evidence each needs, the overrides, and eleven invariants. tests/test_invariants.py checks those invariants on about 1,600 generated conversations in Anthropic's and OpenAI's request shapes (real tool schemas, timestamps in the system prompt, multi-turn tool calls and results), hosted and behind an engine: one change of each kind is named exactly, a full read is never a miss, appending is never a change, the earliest of two changes wins, avoidable tokens stay within what was reusable, engine evidence never overrides a changed prompt, and the hashes ignore exactly key order and volatile values.
• Accuracy at scale, measured (E5). experiments/e5_scale replays conversations through Prefixwise into a live engine with its KV events joined, injects a known change at random turns (tools reordered, a tool added or removed, a timestamp, a system prompt edit, an edited earlier message, an edited tool result), and scores the verdict. On Colab L4 with Qwen3-4B: vLLM 0.33.0, generated conversations 441 of 441 named, real Claude Code trajectories (nvidia/Open-SWE-Traces) 425 of 441; SGLang 0.5.23, generated 456 of 456, real 424 of 434. The 26 not named as the primary cause were edits appended to the latest tool result, which fall in the prompt's last, uncached block: seen as a secondary cause, nothing lost. None was named as another cause, no turn that only appended (3,622) got a change cause, and none of the 5,994 requests failed. Not exercised: an edit at the start of the latest tool result, evictions under memory pressure, other models. Records: experiments/e5_scale/runs/.
• prefixwise import gateway: a gateway's request log, with the live proxy's verdicts. A JSON Lines log (one request per line: its body, the response with usage, when it started and ended; field names matched from the common ones) is replayed through the proxy's own pipeline on the log's clock: start and end events in time order, so requests that overlapped still overlap. Trying Prefixwise on yesterday's traffic needs no change to production. Checked by running the same traffic through the live proxy and the import: identical cause, detail, avoidable tokens and cost on every request, over 40 generated traffics with two interleaved conversations, partial and full reads and gaps past the cache's lifetime. Prompt text is read to fingerprint it and never stored.
• How far a fix would take the hit rate. The report gave avoidable tokens and dollars but not the question teams ask: is there room from here? It now adds, after the list of avoidable causes, "Fixing these would take the share served from cache from 74.6% to 87.1%; the rest is new content each turn, misses with a legitimate cause and misses whose cause is not known", and the shareable block "87% from cache if fixed" (rounded down, with a decimal when whole percents would hide the gain or read as a perfect cache). It is (cached + avoidable) / prompt over the same requests: avoidable tokens are exactly the reusable prefix the cache did not serve on wasteful requests, so a fix moves them and nothing else. Checked against a real run: Claude Sonnet 4.6, 20,089 of 26,932 tokens cached (74.6%) with 3,375 avoidable predicts 87.1%, and the same agent with --optimize read 87.1%.
• prefixwise.wrap records streamed calls. create(..., stream=True) and the SDKs' .stream() helpers (messages.stream and beta.messages.stream on Anthropic, chat.completions.stream and responses.stream on OpenAI), sync and async, were passed through unrecorded; agents that stream every call recorded nothing. The stream the caller iterates is still the SDK's own (isinstance(s, openai.Stream), s.response, context managers and close() all work as before); usage is read from its events as they pass (Anthropic message_start and message_delta, the final usage chunk on Chat Completions, response.completed on Responses) and the call is recorded once, when the stream ends. A stream closed or cancelled before its end is a failure row (client_disconnected), as the proxy records a client that hangs up, the helpers' streams too; one the server or the network breaks (an error event mid-stream) is upstream_dropped; one dropped without being closed or read to its end is not recorded. On Chat Completions usage is only streamed when stream_options.include_usage is set: when the caller has not set it (in stream_options or extra_body), wrap asks for it on a copy of the caller's options and keeps the extra usage-only chunk out of the caller's stream; a server that rejects stream_options gets the call as the caller made it, and is not asked again. The session is the one current when the call was made, wherever the stream is consumed. Streams read through with_raw_response or with_streaming_response are left untouched and unrecorded.
• OpenRouter keeps its sticky routing through Prefixwise. OpenRouter sends a conversation back to the provider that holds its cache by the body session_id (its caching guide). Prefixwise removed a client's body session_id on the way to every hosted API, since most reject unknown fields, so through Prefixwise an agent that sent one lost OpenRouter's stickiness, its cache could miss, and the miss was blamed on routing. A client's own session_id now reaches an upstream on openrouter.ai unchanged (PREFIXWISE_FORWARD_SESSION_ID=1 for another gateway that reads it); other hosted APIs still never get one, and Prefixwise never sends one of its own.
• OpenAI's user groups turns. With no session header or body session_id, all traffic was one session; an end-user id now names it (Anthropic metadata.user_id, as before, or OpenAI user); prefixwise check now ignores both, so its scenarios never merge. It is kept out of OpenTelemetry's conversation id, also when recorded through prefixwise.sdk. Turns of one user's different conversations share that session, and the first turn of a later conversation with the same system prompt can read as a history edit, as it did in the single default session before: send a conversation id where you have one. A prompt_cache_key is deliberately not used: a key that changes mid-conversation is the miss cache_key_changed names.
• The earliest change in the conversation is the one named. With an earlier tool result and a later message both changed, Prefixwise named the later edit (history_edit): it checked all messages before all tool results instead of walking the conversation in the order it was sent. Both are waste, so the avoidable count was right; the cause pointed at the wrong block. Found by the new invariant tests.
• A mid-conversation developer or system message is growth, not a system prompt edit. On OpenAI's chat shape every system and developer message was taken as part of the system prompt, wherever it was sent, so an agent that appends a reminder mid-conversation got system_edit on each such turn (or none with system_edit as a secondary cause). Only the messages that open the conversation are the system prompt now; a later one is a message where it was sent, as the provider renders it. Found by the specification audit.
• A parallel subagent of another type is a new conversation, not an unjudged one. The in-flight rule (a request sent before its previous turn finished is unknown) compared a new conversation with another conversation's turn, so a subagent started while the main agent's request was in flight came out unknown instead of first_write. And no detail is attached to a turn left unjudged: one could promote it to a cause from a negative gap.
• Behind an engine, the "first changed" column follows the engine's prefix order (system before tools), as the cause does; it named the tools when both changed and the cause was the system prompt. A hosted routing_miss now says it is inferred.
• Second audit of the specification and the importer. Three independent reviews read docs/ALGORITHM.md against the code; what they found is fixed here, and the specification now states only what the code does:
  • Nothing is avoidable when the previous turn had no usage. Its prompt size is not known, so how much of this prompt was reusable is not known either; the whole prompt was counted, appended content included.
  • The in-flight rule uses half of the shorter prompt, as the classifier's half-read test does: after a large tool result was appended, a turn that read more than half of the previous prompt could still count as unread, because that was less than half of its own, longer prompt.
  • Dates without a year are masked (Nov 12, 12. November), so a system prompt carrying today's date in that form is volatile_system, not system_edit. Only a capitalized month and a day of 1 to 31 count, so Rule 1 may be skipped or version 2.1 May release are not...

0.11.7

For anyone running Prefixwise in front of vLLM or SGLang: fixes from a review of every vLLM and
SGLang blog post that touches the prefix cache (about 60 of 310), each checked in the engines'
source. Routers now get the routing keys they route by, host_restored is claimed only
on a hit the engine join itself judged, and an SGLang with its radix cache off is no longer an
eviction on every turn. Upgrade the collector first, then the edge.

• Routing keys reach the router. Prefixwise forwarded a fixed list of headers that left out the keys routers and engines route by: x-sgl-routing-key (sgl-router's sticky policy), x-smg-routing-key and x-smg-target-worker (SGLang Model Gateway), x-dynamo-session-id (Dynamo) and x-data-parallel-rank (vLLM and SGLang pin a rank by it). Behind a router, a session could then land on any worker and miss; across data-parallel ranks whose events Prefixwise joins, those misses were confirmed as routing_miss, with the advice to fix routing, for a rank the client had pinned. They now pass through unchanged. To a self-hosted engine or its router, x-forwarded-for and cookie pass too (the Gateway's consistent hashing falls back to them); never to a hosted API, where they could carry a login cookie or an address meant for Prefixwise's own ingress. --pass-header NAME (or PREFIXWISE_PASS_HEADERS) adds a router's own header name; Prefixwise's own headers and those urllib sets are refused there. Behind the Gateway with --engine none, add --pass-header cookie if clients rely on it.
• host_restored only from a hit the engine join judged. With no offload tier at all, a turn that appended a large tool result could be labelled host_restored with "the tier's own stores were seen": the classifier called it a hit (half the shorter prompt read), the join, judging by this prompt, took its miss path, and the blocks it found removed became "restored". Now only the join's own hit verdict can become the detail, on vLLM and SGLang. Reproduced from a recorded event sequence. A new edge and an older collector still claim it the old way; a new collector with an older edge claims it not at all: upgrade both.
• SGLang: a prefix another conversation recomputed is not a restore. The docs said so; the SGLang join did not check (vLLM's did). A store by another session of the previous request's blocks while this one is in flight now withholds host_restored; SGLang's load-backs carry no session, so a real restore still counts.
• A restarted engine must show its tier again. The flag that an offload tier exists was never reset when the engine restarted or flushed its cache, so an engine restarted without a tier could still get host_restored. (After a flush, a tier that restores before its next store of its own is not claimed: conservative.)
• SGLang with its radix cache off. --disable-radix-cache (HiSparse requires it; deterministic inference sets it on most attention backends) means nothing is ever read back, yet an unchanged prefix that missed was an inferred eviction on every turn. /server_info says so, now read with or without KV events, and such a miss is unknown, "the engine keeps no prefix cache", with nothing avoidable. Only for the provider the engine serves (a hosted API behind the same prefixwise run keeps its causes), and with no detail's advice attached. Turn facts gain no_prefix_cache, sent only when true: a 0.11.6 prefixwise collect refuses those records, so upgrade the collector first if you run SGLang with its radix cache off.
• SGLang Mamba models ignore --full-attention. /server_info shows uses_mamba_radix_cache; SGLang frees Mamba state without an event, so the flag would let the events rule out evictions that happened.
• More fields that change the prompt are compared: reasoning_effort (vLLM passes it to the chat template, and writes it into gpt-oss's system message), mm_processor_kwargs and media_io_kwargs, SGLang's lora_path, documents and continue_final_message. A miss where only one changed is param_rendered, not an eviction. cache_salt is not compared yet, because the E1 experiment forces its retry misses with a fresh salt; it is folded into the cross-conversation prefix keys, so tenants with different salts are no longer told they could have shared a prefix.
• Docs: host_restored with an SGLang L3 storage backend. docs/SGLANG.md said the detail means a restore from host memory. With --hicache-storage-backend in the default host memory mode, SGLang publishes a refill from storage as a host (CPU_PINNED) store and does not yet emit storage-tier events (sglang#29709), so there host_restored also covers a restore from storage; in buffer_only mode such a restore is a plain none. The page now says so, and that SGLang's per-request return_cached_tokens_details (not read by Prefixwise yet) is what would tell the two apart. Raised by an SGLang contributor reviewing the HiCache docs recipe (sglang PR #49194), and checked in SGLang's source.

0.11.6

The report now says where the whole bill went, not only what was avoidable: cache re-reads, new input,
cache writes and output. On a long, well-cached session the re-reads are the largest part, and no hit-rate
fix lowers them; the report says so and names the lever. prefixwise import opens with the headline.

• Where the bill went, and the re-read floor. The report showed what was avoidable, and not what the rest of the bill was. It now splits the cost into cache re-reads, new input, cache writes and output (prefixwise report, the shareable block and report --session), and when re-reads are the largest part it says so plainly: that part re-reads context the cache already holds, so a better hit rate would not lower it; the lever is how much context each turn re-reads. report --session also gives the cached tokens each turn that hit re-read, average and largest. From a gateway engineer's question on Reddit, with a DeepSeek session at a 99.7% hit rate where re-reads were 62% of the bill. Ledger schema 7 adds cost_cache_read, cost_input, cost_cache_write and cost_output; rows recorded before it have no split, and the line says how many requests it covers. If you send rows to ClickHouse, add the four columns before upgrading (ALTER TABLE ... ADD COLUMN cost_cache_read Nullable(Float64), and so on; docs/DATA.md): a server that refuses unknown fields drops every batch until then, and one that skips them stores no split.
• prefixwise import opens with the headline. The first screen after importing Claude Code or Codex logs was two paragraphs of caveats before any number. Now it opens with the short block made for pasting into a chat (requests, share served from cache, cost, the top avoidable cause and its fix, unread writes, how to install), then the details, with the caveats at the end. The totals appear once.

0.11.5

For anyone using an OpenAI-compatible API that is not OpenAI (DeepSeek, Tencent, Together, OpenRouter) or
a self-hosted engine: prefixwise serve works with a base URL ending in /v1, and OpenAI's own caching
minimum is no longer applied to them. Checked live on DeepSeek.

• prefixwise serve --upstream <base>/v1 works. Most OpenAI-compatible APIs document a base URL ending in /v1 (DeepSeek, Tencent TokenHub, Together), and serve added a second one: every request went to /v1/v1/chat/completions and came back 404. prefixwise run already handled it. Found by the live DeepSeek run below.
• DeepSeek, checked live. Through its API (deepseek-flash), a repeated 3,000-token prompt read 2,816 tokens from the cache on turns 2 and 3, reported both as OpenAI's prompt_tokens_details.cached_tokens and as DeepSeek's own prompt_cache_hit_tokens, the same number. Prefixwise now also reads DeepSeek's field when the standard one is absent, with prompt_cache_miss_tokens filling in a missing prompt_tokens.
• OpenAI's 1,024-token minimum applies to OpenAI's models only. Any API in OpenAI's request shape was judged by it, so a short prompt that missed on DeepSeek, Tencent, OpenRouter or Together was called below_min_prefix, and so was one on vLLM or SGLang, which cache from a single block. Now the minimum applies to OpenAI model ids (gpt-* except the open-weight gpt-oss, o1/o3/o4, chatgpt-*, codex-*, computer-use-*, fine-tuned ft: ids, and route prefixes such as openrouter/openai/) on a hosted API. On another OpenAI-compatible API, whose minimum Prefixwise does not know, a prompt under 1,024 tokens that missed is unknown, said to be possibly below that API's minimum, not a routing miss or a retry (live, DeepSeek cached nothing of a 239-token prompt sent three times); on an engine no minimum is claimed. From a review of what Tencent's Hy4 (OpenAI-compatible, a cache hit at about 5% of the input price) would get through Prefixwise.

0.11.4

Two fixes from real questions. With SGLang's --enable-session-radix-cache, a client's own body
session_id now reaches the engine, and a session sent only as an x-session-id header is named when
its prefix is evicted. A tool whose schema lists required in another order is now the same tool, for
both the diagnostic and --fix canonical_tools.

• SGLang with --enable-session-radix-cache: a client's body session_id reaches the engine, and a dropped session header is named. That flag makes SGLang evict KV that no session holds first, and SGLang reads the session from the body session_id only. Prefixwise removed a client's body session_id on its way to SGLang unless its own KV-event join was on, which it never is with this flag, so the client's sessions lost their protection; with the flag on (read from /server_info) it now passes it through. Without the flag nothing changes: there it makes no difference to the cache. A client that names its session only in an x-session-id header (which SGLang does not read, and which per sglang#47763 sgl-router does not forward) gets a one-time warning, and an eviction of its prefix gets the detail session_header_dropped, with the fix: send the id as a body session_id too (SGLang PR #45620 would read the header; not merged). It is a detail of evicted, not a cause of its own: the miss is an eviction, and measured live it happens only with that flag on and memory full: 4,000 of 4,012 tokens cached with a body session_id, 3 without; with the flag off, 3 either way (record). Turn facts gain session_unprotected, sent only when true: a 0.11.3 prefixwise collect refuses those records as malformed, so upgrade the collector first if you run SGLang with that flag.
• A tool whose schema lists required in another order is the same tool. JSON Schema's required (and a list type, such as ["string", "null"]) is a set, and a schema built from a Python set lists it in a different order after a restart. Prefixwise called that tool_catalog_change, a legitimate change, and --fix canonical_tools sent it as it came, so the miss was neither counted as waste nor fixed. Now the detector names it tool_reserialize and the fix sorts those two arrays, at any depth; enum, anyOf and every other array keep their order, and default, example and const values are never touched. Key order was already sorted at every depth. From a Reddit question (r/LLMDevs) about nested tool schemas. Such a tool hashes differently from before, so across the upgrade (a restart, or an edge and a collector on different versions) expect one tool change per conversation that uses one; upgrade an edge and its collector together.

0.11.3

One fix for anyone running Prefixwise in front of a router that routes by session: the client's
x-session-id now reaches the router unchanged, and Prefixwise sends no session header of its own.

• The client's x-session-id is forwarded, and Prefixwise sends none of its own. The proxy replaced it with its own session id on every request, default for traffic with no Prefixwise session, so behind a router that routes on that header (sgl-router, llm-d, AIBrix) every session shared one routing key and landed on one worker. Now the client's value goes through unchanged, groups turns into sessions when no Prefixwise id is given, and is the session SGLang's KV events are labelled with; with no client value, none is sent, as docs/ENGINES.md always said.

0.11.2

For local models. On Ollama and most llama.cpp builds, which do not report cached tokens, Prefixwise no
longer calls every unchanged turn a routing miss: it says the count is missing and still names every
change to the prompt. Prefixwise inside vLLM with LMCache is validated live, and prefixwise on its
own shows how to start.

• A server that reports no cached tokens is unknown, not a routing miss. Ollama and many llama.cpp builds send prompt_tokens with no prompt_tokens_details, so their 0 cached means "not reported". Prefixwise read it as nothing reused and called every unchanged turn a routing_miss ("reached a machine without the cache"). Now such a turn is unknown, with that evidence; the report says the server leaves the count out and that the hit rate is not known rather than zero; the session view shows ?. Changes to the prompt (tool order, a timestamp, an edited message) are still named, from the prompt alone. A server that reports 0 is unchanged. Found by running a new user's first session against an Ollama-style server.

• prefixwise with no command shows how to start: the five commands most people need, instead of an argument error. A report before any run says to run an agent through Prefixwise first.

• Prefixwise inside vLLM works with LMCache, live. One flag on vllm serve, no proxy: after a GPU flood every next turn was a hit labelled host_restored, 5 of 5, as through the proxy; the canonical_tools replay on the same server matched the vLLM-alone run to the token; and prefixwise serve detected vLLM and its event stream from the URL alone, its first live record (record). docs/LMCACHE.md gains the one-flag setup.

0.11.1

Documentation only: no change to what the package does. The README is a third of its length and shows a
real run, so the page on PyPI says what Prefixwise does in the first screen.

• The README shows a real run. --optimize on the Anthropic API, before and after, on one small agent: 74.6% to 87.1% served from cache, $0.0356 to $0.0250, 3,375 avoidable tokens to none (record). It replaces the mock output. In that run, the turn that flipped the tool list back to an order still in the cache read almost all of it.
• A shorter README. About 160 lines where there were 510: what Prefixwise does (identify, fix, prevent, auto-detect), the three commands, real output, the results with their records, the ways in, and the limits. Everything it held before is kept whole in docs/GUIDE.md.

0.11.0

--optimize does more. auto_breakpoints passed its live test on Anthropic, so --optimize now places
cache_control markers on an Anthropic request that carries none, once its tools and system prompt were
seen within the cache window: 106.8% of the predicted savings realised and the prompt-side cost down 85.6%,
write surcharge included, with tool choices unchanged. A request Prefixwise marked is never the reason for
an error, and nothing is placed for a self-hosted engine. A minor version, not a patch, because a flag you
may already use changes what it sends; --fix canonical_tools keeps the old behaviour. The GitHub Action's
moving tag is now v0.11 (v0.10 stays at 0.10.7).

• auto_breakpoints is proven, and --optimize turns it on. E4 live on Anthropic (Haiku 5.5), a client that sets no cache_control, 60 paired requests: 106.8% of the predicted savings realised, prompt-side cost down 85.6% with the write surcharge included, tool choices 59 of 60 right without the fix and 60 of 60 with it, no marker rejected (record). What changes for you: with --optimize (or PREFIXWISE_OPTIMIZE=1, or optimize in PREFIXWISE_FIXES), an Anthropic request that carries no marker now gets them once its tools and system prompt were seen within the cache window; requests with their own markers, one-shot requests, other providers and self-hosted engines (the vLLM middleware included) are untouched. To keep the old behaviour, name the fix you want: --fix canonical_tools. The limits are the run's: one model, multi-turn sessions seconds apart, Anthropic's own API.
• A request Prefixwise marked is never the reason for an error. Any 400 or 422 on a request that carries placed markers is resent as the client sent it, whether or not the error names cache_control (a gateway may say only "extra inputs are not permitted"), together with any added field the error names; the markers count as refused only when the resend goes through, and after two such they are no longer placed. Before, only an error naming the field was resent.
• prefixwise check ignores PREFIXWISE_FIXES. The gate judges the agent as it is; a fix exported in the CI shell no longer hides a finding.
• E4 covers a second fix: experiments/e4/e4.py --fix auto_breakpoints replays a client without markers and scores it in tokens and in dollars (prompt side, surcharge included); one request sent without its markers fails the run.