fix(ai): avoid crashing on malformed streamed tool-call input#1707
fix(ai): avoid crashing on malformed streamed tool-call input#1707craze3 wants to merge 2 commits intovercel:mainfrom
Conversation
Signed-off-by: zz <craze3@gmail.com>
🦋 Changeset detectedLatest commit: 54db77b The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@craze3 is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
|
@vercel/workflow ready for review. This fixes #1706 by preserving malformed streamed tool-call input during DurableAgent reconstruction so It looks like the Vercel preview checks on this fork PR are blocked pending team authorization. |
VaguelySerious
left a comment
There was a problem hiding this comment.
LGTM generally, I was concerned about leaving the malformed string in there potentially, so I had AI do a quick assessment. Let me know what you think about it / adjust as you see fit, then feel free to re-tag me for review here or send a comment on the issue
If the repair function fixes the tool call and execution succeeds, the loop continues and the prompt is sent to the provider on the next turn. Providers serialize this input directly:
- OpenAI (@ai-sdk/openai, internal/index.mjs:224): does arguments: JSON.stringify(part.input). If input is already a string like '{"city":"San', JSON.stringify produces a double-encoded
string literal '"{\"city\":\"San"' — not a valid function arguments value. - Anthropic (@ai-sdk/anthropic, internal/index.mjs:2451): passes input: part.input directly as tool_use.input. Anthropic's API expects a JSON object here; a string will be rejected.
So the flow is:
- Model emits malformed tool-call input
- safeParseToolCallInput preserves it as a string (no crash — good)
- String is baked into conversationPrompt
- executeTool in durable-agent.ts:1581 hits JSON.parse → throws → repair runs → tool executes successfully
- Next doStreamStep call sends the contaminated prompt to the provider
- Provider API error on the next turn — a new failure that didn't exist before
Before this PR, the crash at step 2 prevented the loop from ever reaching step 5-6. After the PR, you trade an immediate crash for a deferred one on the next model call. This only
matters when all three conditions hold: malformed input + repair succeeds + there's a subsequent model turn. But that's exactly the scenario the PR is designed to enable.
Suggested fix:
After repair succeeds in executeTool, the repaired tool call input should be patched back into the conversationPrompt. Alternatively, streamTextIterator could defer writing the
assistant tool-call message until after tool execution/repair, using the (potentially repaired) input.
| // Mock doStreamStep | ||
| vi.mock('./do-stream-step.js', () => ({ | ||
| doStreamStep: vi.fn(), | ||
| safeParseToolCallInput: (input: string | undefined) => { |
There was a problem hiding this comment.
I think this can just be original.safeParseToolCallInput instead of rewriting the function, to avoid drift
There was a problem hiding this comment.
Moved the parser into a shared internal helper, so stream-text-iterator now imports the real implementation instead of re-declaring it in the test mock.
| * Parse streamed tool-call input without crashing the workflow step when a | ||
| * provider emits malformed or truncated JSON. | ||
| */ | ||
| export function safeParseToolCallInput(input: string | undefined): unknown { |
There was a problem hiding this comment.
Could this also re-use safeParseInput from durable-agent.ts? Or is the undefined fallthrough intentional?
There was a problem hiding this comment.
Pulled this into a shared helper and now reuse it from do-stream-step, stream-text-iterator, and durable-agent. The undefined/empty-input -> {} behavior is still intentional for no-args tool calls.
Signed-off-by: zz <craze3@gmail.com>
|
Good catch on the deferred-failure path. I pushed a follow-up commit that patches repaired tool-call input back into the accumulated assistant tool-call message before the next model step, and also updates the original tool-call object so downstream I also moved the parser into a shared helper to avoid drift between the touched files and tests. Re-ran the focused pnpm --filter @workflow/ai test -- \
src/agent/do-stream-step.test.ts \
src/agent/stream-text-iterator.test.ts \
src/agent/durable-agent.test.ts \
src/agent/telemetry.test.ts
pnpm --filter @workflow/ai build |
Description
Closes #1706
tool-call.inputJSON.parse(...)calls indo-stream-step.tsandstream-text-iterator.tsthat could crash the workflow step beforeexperimental_repairToolCallwas reachedThis keeps the fix narrowly scoped to internal DurableAgent reconstruction paths. It does not change the public API or introduce provider-specific behavior.
How did you test your changes?
packages/ai/src/agent/do-stream-step.test.tsdoStreamStepwith a mock streamed tool call whose input is malformed JSON and verifies the step no longer throwspackages/ai/src/agent/stream-text-iterator.test.tsthat verifies malformed tool-call input is preserved into the reconstructed assistant message instead of crashing during prompt rebuild@workflow/aitest suite for the touched agent paths:pnpm --filter @workflow/ai test -- \ src/agent/do-stream-step.test.ts \ src/agent/stream-text-iterator.test.ts \ src/agent/durable-agent.test.ts \ src/agent/telemetry.test.tsRecommended commands to rerun in a fresh fork before marking ready:
pnpm --filter @workflow/ai test -- \ src/agent/do-stream-step.test.ts \ src/agent/stream-text-iterator.test.ts \ src/agent/durable-agent.test.ts \ src/agent/telemetry.test.ts pnpm --filter @workflow/ai buildNotes
{}so repair hooks and debugging still have access to the original provider output.PR Checklist - Required to merge
pnpm changesetwas run to create a changelog for this PR@vercel/workflowin a comment once the PR is ready, and the above checklist is complete