fix(agent-runtime): call Anthropic's native Messages API so prompt caching works

[kmonsoe]

Anthropic emailed us that Threll.ai's prompt cache hit rate is low, estimating up to 59% of direct API spend recoverable. It was zero, and the cause was the transport.

The bug

Orgs on the Anthropic provider preset were hitting https://api.anthropic.com/v1/chat/completions — Anthropic's OpenAI SDK compatibility layer, which does not support prompt caching and silently ignores unsupported fields. The runtime has emitted cache_control markers since the caching work landed, and isAnthropicCompatibleBackend() explicitly turned them on for api.anthropic.com — every one of them was dropped on the floor.

It was invisible from our side too: the compat layer returns usage.prompt_tokens_details as always empty, and runtime.ts only accumulated prompt_tokens/completion_tokens, so we had no cache counters anywhere. OpenRouter with an anthropic/* model was the only path where caching actually worked, which is why this went unnoticed — that's the default preset.

The fix

anthropicNativeProvider posts to /messages with x-api-key + anthropic-version, translating the runtime's OpenAI-shaped ChatMessage both ways: system messages hoisted into system blocks, tool_calls → tool_use, role:'tool' → tool_result (parallel results merged into one user message so the model isn't trained out of parallel calls), stop_reason → the runtime's finish reasons. defaultProvider routes api.anthropic.com there and leaves every other backend — OpenRouter, OpenAI, self-hosted — on the compat provider. Dispatch is per-request on base URL, so the metering wrapper in runner.service.ts is unchanged.

Three cache breakpoints per request (max is 4):

Breakpoint	TTL	Why
Last tool	1h	Tool catalog is identical for every conversation in an org
Last non-volatile system block	1h	Org prompt + company profile + channel descriptor, likewise
Newest turn	5m (default)	Each tool-loop iteration reads the previous iteration's prefix

The per-conversation context moves out of the cached prefix. conv appended You are replying in conversationId: <id> to the end of the system prompt, which made the entire prefix vary per conversation and never share an entry. It now travels as AgentConfig.volatileSystemPrompt, is emitted as a trailing block flagged ChatMessage.volatile (as is the history-truncation note), and the breakpoint lands after the last non-volatile block. This helps the OpenRouter path too, whose breakpoint sits on the first system block and was carrying the conversation id with it.

Two things surfaced while building that weren't in the original diagnosis:

• max_tokens is required on the native API and nothing in the conversation path sets it — defaults to 8192.
• Adaptive thinking is on by default on current models, and dropping thinking blocks from an assistant turn makes the follow-up carrying tool results fail. Assistant turns now round-trip their raw blocks via ChatMessage.providerContentBlocks and are echoed byte-for-byte; breakpoints only ever attach to text/tool_result blocks, never to a block we echo verbatim.

Also drops api.anthropic.com from the compat provider's cache auto-detection — the source of the false confidence — and moves the shared rate-limit retry and ProviderError classification into providers/transport.ts so both providers use one implementation.

Verified against the live API

claude-haiku-4-5, real key:

turn 1                     prompt=7820  cache_write=7817  cache_read=0
turn 2 (tool result)       prompt=7927  cache_write=105   cache_read=7817
turn 3 (new conversation)  prompt=7816  cache_write=346   cache_read=7467

Turn 1 writes the prefix (previously always 0). Turn 2 reads all 7817 and writes only the 105-token incremental turn — the tool-loop win. Turn 3, on a different conversationId, still reads 7467 of the shared prefix, which is the volatile-block split working; before it, that read would have been 0. The tool round-trip is confirmed end-to-end (Haiku called kb_search, the tool_result was accepted, the reply used it correctly).

Four shapes the native API validates strictly and the compat layer tolerated — history opening with an assistant turn, assistant-only history, empty message bodies, no system blocks — all accepted.

Automated: 514 agent-runtime + 73 agent-host tests, typecheck and lint clean. 27 new tests cover request shape, headers, breakpoint placement and TTLs, the stable/volatile boundary, the thinking echo, parallel tool-result merging, assistant-first history, stop-reason and usage mapping, and 401 → provider_auth.

Not verified

The thinking-block echo never triggered live. Haiku 4.5 doesn't think by default; Sonnet 5 declined on a trivial task; Opus 5 returned no thinking blocks even on a reasoning-heavy prompt. With our request shape — no thinking parameter, so display defaults to omitted — the API returns none at all. So providerContentBlocks is verified to round-trip what the API actually returns (text + tool_use, byte-for-byte), but the thinking case is defensive rather than exercised. It only becomes load-bearing if we opt into thinking: {display: 'summarized'}, which nothing in AgentConfig can currently request.

Follow-ups (not in this PR)

• ProviderUsage now carries cache_read_input_tokens / cache_creation_input_tokens, but nothing reads them. Threading them into AgentReply and per-org usage is what would let us confirm the hit rate from the dashboard instead of the Console.
• The audit pass's JSON mode stays off on Anthropic (native has no response_format; audit.ts already skipped it and parseVerdict handles prose-wrapped JSON). Structured outputs would be a genuine upgrade there.

🤖 Generated with Claude Code