Summary
When using OpenRouter with a Claude model, _anthropic_prompt_cache_policy() correctly returns (True, False) — signaling that caching should be applied with the "envelope" layout. However, the cache markers are never transmitted to Anthropic because OpenRouter routes Claude requests via the OpenAI-wire format (chat_completions api_mode), which sends the system prompt as a {"role": "system", ...} message inside the messages[] array. Anthropic's caching infrastructure only honors cache_control on top-level system content blocks in the native anthropic_messages wire format — it ignores them on the OpenAI-compat path.
The result: every request through OpenRouter bills full input tokens (16-19K for the default system prompt), even though the code believes caching is active.
Root Cause
run_agent.py api_mode selection (line ~1088):
elif self.provider == "anthropic" or ...:
self.api_mode = "anthropic_messages"
elif self._base_url_lower.rstrip("/").endswith("/anthropic"):
self.api_mode = "anthropic_messages"
else:
self.api_mode = "chat_completions" # <-- OpenRouter lands here
OpenRouter's base URL is https://openrouter.ai/api/v1 — it does not end in /anthropic, so it gets chat_completions. This is correct for the wire format (OpenRouter speaks OpenAI-compat).
_anthropic_prompt_cache_policy() (line ~2967):
python
if is_openrouter and is_claude:
return True, False # should_cache=True, native_layout=False
should_cache=True causes apply_anthropic_cache_control() to inject cache_control markers on message envelopes. But because api_mode == "chat_completions", those messages are sent as {"role": "system", ...} inside messages[] — not as top-level system content blocks. OpenRouter passes this through to Anthropic verbatim, and Anthropic never sees valid caching markers.
Confirmation: A live request dump with ANTHROPIC_BETA headers logging shows no anthropic-beta: prompt-caching-2024-07-31 header and no cache_control in the body — the markers are added to the Python objects but lost when serialized into the OpenAI-wire body.
Expected Behavior
For OpenRouter + Claude, one of:
Option A (preferred): OpenRouter supports Anthropic's native system + cache_control passthrough on their /api/v1/messages endpoint (undocumented but reportedly works). If Hermes uses that endpoint (or passes system as a top-level key with content blocks), caching activates. Implement a special-cased request build path for openrouter + claude that constructs a hybrid request: system as a list of content blocks with cache_control, messages in OpenAI format, and adds the anthropic-beta: prompt-caching-2024-07-31 header.
Option B (defensive): Change _anthropic_prompt_cache_policy() to return (False, False) for OpenRouter+Claude, removing the false claim of caching support. Saves the CPU of injecting markers that do nothing and stops misleading any instrumentation that checks _use_prompt_caching.
Option C (future): Route OpenRouter+Claude through anthropic_messages api_mode directly (using the OpenRouter /api/v1/messages endpoint) when the model is Claude. This would enable full native caching support at the cost of maintaining an OpenRouter-specific routing branch.
Impact
- Every call with OpenRouter + Claude bills full 16-19K input tokens with no caching discount
- System prompt is static and identical across calls — it is the ideal caching target
- At ~$3/MTok (claude-sonnet-4-6 via OpenRouter), a 19K token system prompt costs ~$0.057/call with no caching vs ~$0.006/call with caching active (10x difference)
- For gateway users with frequent short queries, this is the dominant cost driver
Reproduction
python
config.yaml: provider: openrouter, model: anthropic/claude-sonnet-4-6
Observe in run_agent.py after init:
agent = AIAgent(provider="openrouter", model="anthropic/claude-sonnet-4-6", ...)
print(agent.api_mode) # "chat_completions"
print(agent._use_prompt_caching) # True <-- misleading
print(agent._use_native_cache_layout) # False
Actual request body (captured via HTTPX debug logging):
{
"model": "anthropic/claude-sonnet-4-6",
"messages": [{"role": "system", "content": "..."}, ...],
...
}
No "system" top-level key. No cache_control in any block. No anthropic-beta header.
Environment
- Hermes: 53a024994 (main, 2026-05-06)
- Provider: OpenRouter (openrouter.ai/api/v1)
- Model: anthropic/claude-sonnet-4-6
- Platform: macOS (Tailscale node, shared profile)
- Relevant files: run_agent.py:_anthropic_prompt_cache_policy(), agent/prompt_caching.py, agent/transports/chat_completions.py
