Repository navigation
0.11.5
For anyone using an OpenAI-compatible API that is not OpenAI (DeepSeek, Tencent, Together, OpenRouter) or
a self-hosted engine: prefixwise serve works with a base URL ending in /v1, and OpenAI's own caching
minimum is no longer applied to them. Checked live on DeepSeek.
prefixwise serve --upstream <base>/v1works. Most OpenAI-compatible APIs document a base URL ending in/v1(DeepSeek, Tencent TokenHub, Together), andserveadded a second one: every request went to/v1/v1/chat/completionsand came back 404.prefixwise runalready handled it. Found by the live DeepSeek run below.- DeepSeek, checked live. Through its API (
deepseek-flash), a repeated 3,000-token prompt read 2,816 tokens from the cache on turns 2 and 3, reported both as OpenAI'sprompt_tokens_details.cached_tokensand as DeepSeek's ownprompt_cache_hit_tokens, the same number. Prefixwise now also reads DeepSeek's field when the standard one is absent, withprompt_cache_miss_tokensfilling in a missingprompt_tokens. - OpenAI's 1,024-token minimum applies to OpenAI's models only. Any API in OpenAI's request shape was judged by it, so a short prompt that missed on DeepSeek, Tencent, OpenRouter or Together was called
below_min_prefix, and so was one on vLLM or SGLang, which cache from a single block. Now the minimum applies to OpenAI model ids (gpt-*except the open-weightgpt-oss,o1/o3/o4,chatgpt-*,codex-*,computer-use-*, fine-tunedft:ids, and route prefixes such asopenrouter/openai/) on a hosted API. On another OpenAI-compatible API, whose minimum Prefixwise does not know, a prompt under 1,024 tokens that missed isunknown, said to be possibly below that API's minimum, not a routing miss or a retry (live, DeepSeek cached nothing of a 239-token prompt sent three times); on an engine no minimum is claimed. From a review of what Tencent's Hy4 (OpenAI-compatible, a cache hit at about 5% of the input price) would get through Prefixwise.