Skip to content

fix: only add cache_control to last tool in Anthropic prompt caching#453

Merged
mikehostetler merged 1 commit into
agentjido:mainfrom
tomtrin:fix/anthropic-cache-last-tool-only
Feb 27, 2026
Merged

fix: only add cache_control to last tool in Anthropic prompt caching#453
mikehostetler merged 1 commit into
agentjido:mainfrom
tomtrin:fix/anthropic-cache-last-tool-only

Conversation

@tomtrin
Copy link
Copy Markdown
Contributor

@tomtrin tomtrin commented Feb 22, 2026

Summary

  • maybe_cache_tools/2 was adding cache_control: %{type: "ephemeral"} to every tool definition via Enum.map, exceeding Anthropic's 4 cache_control breakpoint limit with 3+ tools
  • Since Anthropic's caching is prefix-based, only the last tool needs a breakpoint — it caches all preceding tools in the prefix
  • Changed to use Enum.split(tools, -1) to only annotate the last tool

Before (5 tools + system + message = 7 breakpoints)

tool_a: cache_control ← redundant
tool_b: cache_control ← redundant
tool_c: cache_control ← redundant
tool_d: cache_control ← redundant
tool_e: cache_control
system: cache_control
message: cache_control
Total: 7 (exceeds limit of 4)

After (last tool + system + message = 3 breakpoints)

tool_a:
tool_b:
tool_c:
tool_d:
tool_e: cache_control
system: cache_control
message: cache_control
Total: 3 (within limit of 4)

Test plan

  • Added test with 5 tools verifying only the last gets cache_control
  • All 31 existing prompt cache tests pass unchanged

Fixes #452

maybe_cache_tools was adding cache_control to every tool definition,
quickly exceeding Anthropic's 4-breakpoint limit. Since caching is
prefix-based, only the last tool needs a breakpoint to cache all tools.

Fixes agentjido#452

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mikehostetler mikehostetler merged commit 8d64cd8 into agentjido:main Feb 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Anthropic prompt caching adds cache_control to every tool instead of last tool only

2 participants