Fix Claude prompt caching to preserve 1h TTL for cache_control blocks

[NakiriYuuzu]

Summary

CLIProxyAPI already injects and repairs Anthropic cache_control breakpoints for Claude requests, but the current behavior does not reliably preserve the 1-hour prompt cache path. Injected or default cache_control: { "type": "ephemeral" } blocks remain default 5-minute TTL blocks, and normalizeCacheControlTTL() then treats them as 5m anchors. This can downgrade later ttl: "1h" blocks or prevent newly injected breakpoints from using the extended 1h cache TTL.

The desired behavior is to make Claude requests consistently use 1h cache TTL when the request/header context supports it, while still preserving Anthropic's max-4 breakpoint limit and TTL ordering rules.

Current behavior

The Claude executor currently does these steps:

1. If there are no cache breakpoints, it calls ensureCacheControl().
2. ensureCacheControl() injects cache_control into:
   • the last tool definition,
   • the last system prompt element,
   • the second-to-last user turn when applicable.
3. Injected blocks use only { "type": "ephemeral" }, so they default to 5m.
4. enforceCacheControlLimit(..., 4) removes excess breakpoints.
5. normalizeCacheControlTTL() strips later ttl: "1h" values after any default 5m block in Anthropic evaluation order: tools -> system -> messages.

This means a request can be structurally cacheable but still not use the 1h TTL path.

Relevant code:

• internal/runtime/executor/claude_executor.go
  • ExecuteStream / Execute call ensureCacheControl() only when countCacheControls(body) == 0.
  • ensureCacheControl() injects default ephemeral blocks.
  • normalizeCacheControlTTL() correctly protects Anthropic ordering, but default blocks currently force later 1h blocks to be downgraded.
• internal/runtime/executor/caching_verify_test.go
  • tests injection exists, but does not appear to assert ttl: "1h" on injected blocks.
• internal/runtime/executor/claude_executor_test.go
  • tests TTL normalization and max-4 behavior, but the desired 1h promotion behavior should be covered explicitly.

Proposed fix

Add a dedicated Claude cache-control repair step that runs after injection and max-4 enforcement, before final upstream body signing / sending:

1. Promote eligible default ephemeral cache blocks to ttl: "1h".

   • Eligible means cache_control is an object, type == "ephemeral", and ttl is missing or currently "5m".
   • Do not mutate malformed/non-object cache_control values.
   • Do not overwrite unsupported or unknown TTL values other than missing/"5m" unless maintainers intentionally choose to normalize them.

2. Preserve Anthropic ordering constraints.

   • After promotion, re-run the existing ordering normalization so no 1h block appears after a remaining default/invalid 5m-equivalent block in evaluation order.
   • Keep the current evaluation order: tools -> system -> messages.

3. Keep max-4 breakpoints.

   • Continue to enforce the 4-breakpoint limit.
   • Prefer preserving high-value breakpoints: last tool, last system, and recent message cache points.

4. Keep CCH/body signing correct.

   • If experimental CCH signing is enabled, sign after all body mutation, including cache-control injection, promotion, max-4 enforcement, and TTL normalization.
   • This avoids stale CCH fingerprints after body mutation.

A safe helper shape would be something like:

func promoteDefaultCacheControlTTL(payload []byte, ttl string) []byte {
    // Walk tools -> system -> messages.
    // For each cache_control object where type == "ephemeral" and ttl is missing or "5m",
    // set ttl to "1h".
    // Leave malformed/non-object cache_control values untouched.
}

Then the Claude request pipeline can become:

if countCacheControls(body) == 0 {
    body = ensureCacheControl(body)
}
body = enforceCacheControlLimit(body, 4)
body = promoteDefaultCacheControlTTL(body, "1h")
body = normalizeCacheControlTTL(body)
// CCH signing, if enabled, must happen after this point.

If maintainers prefer configuration, make it opt-in, for example:

providers:
  claude:
    cache-control-default-ttl: "1h"

Defaulting to 1h is probably reasonable for Claude Code-compatible traffic because the project already forwards/uses Claude Code prompt-caching beta behavior and has cache-control repair logic.

Acceptance criteria

• When a Claude request has no cache_control, CLIProxyAPI injects cache breakpoints and those injected breakpoints use ttl: "1h" where valid.
• When a Claude request has existing cache_control: { "type": "ephemeral" } blocks, they are promoted to ttl: "1h" where valid.
• Existing ttl: "1h" blocks remain unchanged when no ordering violation exists.
• If a default/invalid 5m-equivalent block remains before a later 1h block, the later 1h block is still downgraded or repaired so Anthropic does not reject the request.
• Total cache_control breakpoint count remains <= 4.
• Experimental CCH signing, when enabled, signs the final mutated body, not a pre-repair body.
• Tests cover stream, non-stream, and count-tokens paths if all three mutate Claude request bodies independently.

Suggested tests

Add or update tests around these cases:

1. ensureCacheControl() with tools + system + multi-turn messages injects ttl: "1h" on the inserted breakpoints.
2. Existing default blocks are promoted:

{
  "tools": [{ "name": "t1", "cache_control": { "type": "ephemeral" } }],
  "system": [{ "type": "text", "text": "s1", "cache_control": { "type": "ephemeral" } }],
  "messages": [{ "role": "user", "content": [{ "type": "text", "text": "u1", "cache_control": { "type": "ephemeral" } }] }]
}

Expected: all valid cache-control objects include "ttl": "1h" after repair.

3. Max-4 still wins when there are too many breakpoints.
4. Ordering repair still wins when a malformed/default 5m-equivalent block appears before a later 1h block.
5. CCH signing test: final signed body verifies after cache-control promotion.

Reference implementation behavior

In my own Claude proxy path, the stable approach is:

• collect cache-control refs in Anthropic evaluation order: tools -> system -> messages,
• enforce max 4 breakpoints,
• promote default ephemeral blocks to ttl: "1h",
• normalize ordering so later 1h blocks do not appear after default/invalid 5m-equivalent blocks,
• sign the final body after all mutations.

This has been enough to keep Claude prompt caching stable on the 1h path while still respecting Anthropic's cache-control constraints.