Repository navigation
opencode-cache-manager: keep OpenCode's prompt cache warm, including through OmniRoute #19079
cris005
started this conversation in
Show and tell
Replies: 2 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I run OpenCode through OmniRoute, and every time I stepped away for more than 5 minutes the Claude prompt cache expired. The next prompt then paid 1.25x to rewrite the whole context instead of about 0.1x to read it. I wrote an OpenCode plugin to fix that:
https://github.com/noppu-labs/opencode-cache-manager
You pick a cache TTL per session:
5m,1horhybrid(on Claude, 1h on tools and the system prompt, 5m on messages). You can switch it with/cache-manageror set it per model variant. When a session goes idle, the plugin sends a hidden ping just before the cache expires, so it's still warm when you come back. A ping costs roughly one cache read, and it stops after a set number of pings (5 by default). The sidebar shows the cache status, how much of the prompt is cached, when the next ping goes out and what the pings have saved.A few things specific to OmniRoute:
claudeor starting withanthropic/is managed as Claude, GPT-5.6+ is matched by name, and you can map aliases withcacheProfiles. Other models are never rewritten or pinged; the sidebar just shows how much of their prompt was cached./chat/completionsand/responses.1handhybridit addsextended-cache-ttl-2025-04-11to theanthropic-betaheader, merged with any values already there. Some routes need it before the 1h TTL applies.prompt_cache_breakpointto get through the Chat Completions translator. I fixed that in fix(translator): keep OpenAI prompt_cache_breakpoint on content parts for openai/openrouter #18255 (merged 2026-10-27), but no release includes it yet; v3.8.52 is the latest. Until one does, use5mfor GPT sessions through OmniRoute. Claude isn't affected.Install:
There's a sample config for OpenAI-compatible proxies in
docs/examples/. I'd like to hear from anyone on routes or combos I haven't tried.All reactions