Skip to content

fix (ai): serialize parallel tool results in tool-call order#16572

Open
Kripu77 wants to merge 1 commit into
vercel:mainfrom
Kripu77:fix/tool-result-call-order
Open

fix (ai): serialize parallel tool results in tool-call order#16572
Kripu77 wants to merge 1 commit into
vercel:mainfrom
Kripu77:fix/tool-result-call-order

Conversation

@Kripu77

@Kripu77 Kripu77 commented Jul 2, 2026

Copy link
Copy Markdown

Background

Fixes #16567.

When an assistant step makes parallel tool calls, results are recorded in
completion order (whichever execute resolves first). toResponseMessages
serializes that arrival order verbatim, so the same logical step produces
differently-ordered tool-result parts depending on tool timing.

Providers pair tool calls and results by id, so responses are unaffected —
but provider prompt caching is byte-sensitive. A conversation served live
(completion order) and the same conversation rebuilt from persisted UIMessages
(call order, since tool parts are created at call time) serialize differently,
invalidating the entire cached prefix behind the first parallel batch. Measured
on a production agent loop: a 4-tool batch serialized in the exact reverse of
call order re-wrote 26 byte-identical cached blocks on the next request.

Summary

Sort the tool message's tool-result parts to match the order of the
corresponding tool-call parts. Results without a matching call sort last,
tie-broken by id, so the ordering is total and deterministic. Sorting is
skipped when the tool message contains tool-approval-response parts, since
approval responses and their synthesized execution-denied results carry
position-dependent semantics.

Within-step result order is not semantically meaningful to providers, so this
is behavior-preserving — it only makes serialization deterministic and
cache-friendly.

Manual Verification

Validated end-to-end against Bedrock Anthropic prompt caching: with this
change, a 36-request agent session with parallel batches up to size 8 produced
zero cache-prefix invalidations (previously invalidated on every fresh request
after a parallel batch).

Checklist

  • All commits are signed (PRs with unsigned commits cannot be merged)
  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Related Issues

Fixes #16567

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel tool results are serialized in completion order, silently breaking provider prompt caching

1 participant