Repeated read-only tool results are re-sent in full, costing tokens for bytes the model already has

[dwin-gharibi]

Overview

Nothing stops a read-only tool's output from being sent to the model again, in full, when it has
not changed.

An agent working in a repository re-reads the same file across turns, re-lists the same directory
after every edit, re-runs the same lsp_diagnostics. Each repeat pays the full token cost of the
payload, and each repeat also consumes context window — which pulls compaction forward and
therefore costs a summarisation call too.

The adjacent machinery exists but does not address this:

Mechanism	What it does	Why it doesn't help here
limit_large_tool_results (pkg/hooks/builtins/limit_large_tool_results.go)	Truncates oversized results	Bounds a single result; a 40 KiB result repeated five times still costs 5 × 40 KiB
pkg/cache	Caches whole-agent question → answer	Session-level Q&A, not tool-level
Compaction	Summarises history	Reactive, lossy, and costs a model call

Motivation

The obvious fix — cache tool results and skip re-execution — introduces exactly the failure mode
that makes caches dangerous in an agent: the agent reads a file, someone (or the agent's own
shell toolset, or a build step) changes it, and the agent is handed the old contents while
believing it looked. Getting invalidation right would mean tracking every path each tool touches,
including tools whose arguments do not name paths at all.

That risk is not worth the I/O saving, and the I/O is not where the cost is — the tokens are.

Use cases

No response

Proposed solution

Elide repeated output instead of caching it. Concretely: let the tool run every time, hash its
fresh output, and only replace the payload with a short marker when it is byte-for-byte identical
to what the model was already shown for the same arguments in the same session.

[docker-agent] The read_file tool ran and returned output byte-for-byte identical to its earlier
result for these same arguments in this session, so the 41,232-byte payload is not repeated here.
Nothing has changed since you last saw it.

This cannot serve stale data, by construction:

• the tool always executes — there is no stored payload to go stale;
• the comparison is against the tool's fresh output, so a one-byte change means the full new
  output passes through untouched;
• there is no expiry to tune and no invalidation to get wrong.

The saving is in tokens, not latency. That should be stated plainly rather than sold as a cache.

Alternatives

No response

Related issues

No response

Additional context

• Only tools declaring ReadOnlyHint in their annotations. A tool with side effects may return
  identical output for two calls that each did something, so eliding the second would hide a real
  event. 46 built-in tools already declare it, and MCP servers can too (tools.ToolAnnotations is
  mcp.ToolAnnotations).
• Never elide an error result: a repeated identical failure is itself information.
• Never elide a payload smaller than the marker — that would cost tokens rather than save them.
• Opt-in. It changes what the model sees, so it should not be switched on for everyone by default.