Skip to content

v20260622-nightly

@gnachman gnachman tagged this 21 Jun 23:19
Prompt caching previously marked only the system block and the last
tool definition, so the conversation history (the bulk of an
orchestration request, dominated by accumulated tool results) was
re-sent uncached on every turn and billed at full input rate.

Add a rolling ephemeral cache breakpoint on the last content block of
the last message. Anthropic reads the longest matching prefix and
excludes cache_control from the cache key, so the block marked this
turn becomes a cache read on the next turn once a newer turn is
appended after it. String content is promoted to a one-element text
block so it has somewhere to carry the marker; array-form messages
(tool results) are marked in place and stay byte-stable across turns.

To support this, message content blocks gain an optional cache_control
field (omitted from the wire when nil, so unmarked blocks serialize
unchanged), and AnthropicCacheControl is lifted to file scope so both
the system/tools breakpoints and the message breakpoint share it.

The live prompt-caching test asserted call 2 re-created zero cache;
that held only when system+tools were the sole cached segment. With
the rolling marker, call 2 legitimately writes a small entry for its
changed last message while reading the shared prefix, so the
assertion now requires the read to dominate the tail write.

Regenerate the Anthropic live-test cassettes: every Anthropic request
body changed, so the prior recordings no longer match and are removed
in favor of freshly recorded ones (the safety benchmarks included).
Non-Anthropic cassettes are untouched.
Assets 2
Loading