Fix malformed JSON in Anthropic code-execution deltas - #6918
Conversation
🦋 Changeset detectedLatest commit: 5650b8a The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The test is missing stream termination events that the existing test patterns consistently include. The implementation fix is also not yet on this branch.
Reviewed changes — a single new reproduction test for the malformed code-execution delta bug.
- Reproduction test — sets up a mocked SSE stream with a
server_tool_use/bash_code_executioncontent block and asserts the emittedtool-params-deltacontains valid JSON with thetypediscriminator as a string.
⚠️ Missing stream termination events
The test SSE events end after content_block_delta. The existing stream-text test in AnthropicLanguageModel.test.ts consistently includes content_block_stop, message_delta, and message_stop after the delta. While Stream.takeUntil at AnthropicClient.ts:309 may allow the stream to end when the response body is consumed, the message_stop event is the canonical termination signal and content_block_stop emits tool-params-end and processes final params. Omitting these events is inconsistent with the existing test patterns and could lead to incomplete or flaky stream behavior.
Technical details
# Missing stream termination events
## Affected sites
- `packages/ai/anthropic/test/AnthropicMalformedCodeDeltaAudit.test.ts:54` — after the last SSE event, add `content_block_stop`, `message_delta`, and `message_stop` events matching the existing test pattern at `AnthropicLanguageModel.test.ts:57-76`
## Required outcome
- The test should include `content_block_stop` (index 0), `message_delta` (with `stop_reason: "tool_use"`), and `message_stop` events to properly terminate the stream.⚠️ Missing text_editor_code_execution coverage
The bug at AnthropicLanguageModel.ts:2575-2576 affects both bash_code_execution and text_editor_code_execution provider names, but the test only covers bash_code_execution. Consider adding a second test case or parameterizing the existing one to cover both code paths.
DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏
fubhy
added
the
audit
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — what changed since the prior Pullfrog review at 9e605f9:
- Delta interpolation fix — wrapped
contentBlock.providerNameinJSON.stringify()so thetypediscriminator is a quoted JSON string, and removed the spurious trailing}from the template literal (thedelta.substring(1)already contributes the closing brace). providerExecutedfinal-params fix — theproviderExecuted === truebranch now parsesfinalParamsviaTool.unsafeSecureJsonParse, matching the symmetry of thetransformToolCallParamsbranch which already parsed it.- Test consolidation and expansion — the standalone audit test file was removed; the tests now live inline in
AnthropicLanguageModel.test.tsas parameterizedfor-loop cases covering bothbash_code_executionandtext_editor_code_execution. Each test asserts valid JSON in the streamed delta and in the accumulated final tool-call params, and the mock SSE stream includescontent_block_stop,message_delta, andmessage_stoptermination events. - Changeset — patch-level entry for
@effect/ai-anthropic.
All three prior review threads (stream termination, text_editor_code_execution coverage, SSE helper pattern) are retired as addressed.
DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
2 participants
Summary
The first streamed code-execution parameter delta is invalid JSON and poisons both incremental parsing and the accumulated final parameters.
Important
This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.
Anthropic code-execution deltas contain malformed JSON
Module:
ai/anthropic/AnthropicLanguageModelAudit ID:
adapters-ai-2-anthropic-malformed-code-deltaSeverity / confidence: high / high
What happens
The first streamed code-execution parameter delta is invalid JSON and poisons both incremental parsing and the accumulated final parameters.
Why it happens
The rewrite interpolates providerName without JSON quotes, producing values such as {"type":bash_code_execution,...}.
Expected behavior
tool-params-delta must contain incremental JSON, and the code-execution type discriminator is a JSON string.
Relevant implementation
These links and excerpts are pinned to audit base
c9b56ab507f224426ee8388dc450da447ec4715f.packages/ai/anthropic/src/AnthropicLanguageModel.ts:2573-2588View problematic code at
packages/ai/anthropic/src/AnthropicLanguageModel.ts:2573-2588View exact lines on GitHub
Reproduction
pnpm test --run packages/ai/anthropic/test/AnthropicMalformedCodeDeltaAudit.test.tsObserved failure: The emitted delta begins with malformed JSON such as {"type":bash_code_execution,..., causing JSON.parse to throw.
Implementation handoff
The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.
pnpm test --run packages/ai/anthropic/test/AnthropicMalformedCodeDeltaAudit.test.tsAudit provenance
c9b56ab507f224426ee8388dc450da447ec4715f8f9499f562729f5f7b08d8bcc4db86b4aeff8a21adapters-ai-2-anthropic-malformed-code-deltaCloses EFF-340