Emit specialized OpenAI tool results only once by fubhy · Pull Request #6921 · Effect-TS/effect

fubhy · 2026-08-03T18:19:56Z

Summary

apply_patch, shell, and local_shell results are sent once in their specialized output form and again as incompatible generic function_call_output items.

Important

This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.

Specialized OpenAI tool results are emitted twice

Module: ai/openai/OpenAiLanguageModel
Audit ID: adapters-ai-9-openai-duplicate-specialized-tool-output
Severity / confidence: high / high

What happens

apply_patch, shell, and local_shell results are sent once in their specialized output form and again as incompatible generic function_call_output items.

Why it happens

After pushing a specialized output, control falls through to the unconditional generic function_call_output path.

Expected behavior

Specialized Responses API call types have distinct corresponding output item types.

Relevant implementation

These links and excerpts are pinned to audit base c9b56ab507f224426ee8388dc450da447ec4715f.

packages/ai/openai/src/OpenAiLanguageModel.ts:1126-1160

View problematic code at packages/ai/openai/src/OpenAiLanguageModel.ts:1126-1160

            if (Predicate.isNotUndefined(applyPatchTool) && toolName === "apply_patch") {
              messages.push({
                id,
                type: "apply_patch_call_output",
                call_id: part.id,
                ...(part.result as any)
              })
            }

            if (Predicate.isNotUndefined(shellTool) && toolName === "shell") {
              messages.push({
                id,
                type: "shell_call_output",
                call_id: part.id,
                output: part.result as any,
                ...(Predicate.isNotNull(status) ? { status } : {})
              })
            }

            if (Predicate.isNotUndefined(localShellTool) && toolName === "local_shell") {
              messages.push({
                id,
                type: "local_shell_call_output",
                call_id: part.id,
                output: part.result as any,
                ...(Predicate.isNotNull(status) ? { status } : {})
              })
            }

            messages.push({
              type: "function_call_output",
              call_id: part.id,
              output: JSON.stringify(part.result),
              ...(Predicate.isNotNull(status) ? { status } : {})
            })

View exact lines on GitHub

Reproduction

pnpm test --run packages/ai/openai/test/OpenAiDuplicateSpecializedToolOutputAudit.test.ts

Observed failure: The specialized result produced both apply_patch_call_output and function_call_output.

Implementation handoff

The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.

Start with the pinned implementation excerpts and the Why it happens analysis above.
Change the implementation so it satisfies the stated Expected behavior; do not weaken or remove the reproduction assertions.
Run the focused reproduction command(s) and confirm the observed failures become passing tests:

pnpm test --run packages/ai/openai/test/OpenAiDuplicateSpecializedToolOutputAudit.test.ts

Run the affected package's existing tests, then the repository lint and type checks before requesting review.

Audit provenance

Audit base: c9b56ab507f224426ee8388dc450da447ec4715f
Reproduction base: 8f9499f562729f5f7b08d8bcc4db86b4aeff8a21
Findings: adapters-ai-9-openai-duplicate-specialized-tool-output
Initial patch: focused reproduction tests; implementation fix pending

Closes EFF-338

changeset-bot · 2026-08-03T18:20:01Z

🦋 Changeset detected

Latest commit: 8522a9f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages

Name	Type
@effect/ai-openai	Patch
effect	Patch
@effect/opentelemetry	Patch
@effect/platform-browser	Patch
@effect/platform-bun	Patch
@effect/platform-deno	Patch
@effect/platform-node-shared	Patch
@effect/platform-node	Patch
@effect/vitest	Patch
@effect/ai-anthropic	Patch
@effect/ai-openai-compat	Patch
@effect/ai-openrouter	Patch
@effect/atom-react	Patch
@effect/atom-solid	Patch
@effect/atom-vue	Patch
@effect/sql-clickhouse	Patch
@effect/sql-d1	Patch
@effect/sql-libsql	Patch
@effect/sql-mssql	Patch
@effect/sql-mysql2	Patch
@effect/sql-pg	Patch
@effect/sql-pglite	Patch
@effect/sql-sqlite-bun	Patch
@effect/sql-sqlite-do	Patch
@effect/sql-sqlite-node	Patch
@effect/sql-sqlite-react-native	Patch
@effect/sql-sqlite-wasm	Patch
@effect/docgen	Patch
@effect/doctest	Patch
@effect/openapi-generator	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pullfrog

Important

The implementation fix is not yet on this branch. The reproduction test captures the bug correctly but the source change to fix it is absent. Per the PR description, the fix needs to be added to this branch before merging.

Reviewed changes

Reproduction test for duplicate specialized tool output: A new test captures the outgoing HTTP request body and verifies that apply_patch results emit only the specialized apply_patch_call_output item, not also a generic function_call_output.

⚠️ Missing implementation fix

The reproduction test in packages/ai/openai/test/OpenAiDuplicateSpecializedToolOutputAudit.test.ts is correct and will fail against the current source. However, the fix to packages/ai/openai/src/OpenAiLanguageModel.ts lines 1126–1160 is not present on this branch. Each specialized block (1126–1133 for apply_patch, 1135–1143 for shell, 1145–1153 for local_shell) needs a continue to prevent fall-through to the unconditional function_call_output push at line 1155.

Technical details

# Missing `continue` after specialized output blocks

## Affected sites
- `packages/ai/openai/src/OpenAiLanguageModel.ts:1126-1160` — three independent `if` blocks for specialized tool outputs are followed by an unconditional `function_call_output` push

## Required outcome
- Each specialized block must `continue` to the next iteration after pushing its specialized message, skipping the generic `function_call_output` on line 1155

⚠️ Missing tests for `shell` and `local_shell`

The same bug affects shell and local_shell results identically, but the test file only covers apply_patch. Add parallel test cases for these two tool types to ensure complete regression coverage.

^{｜ Fix it ➔ ｜ View workflow run ｜ Using DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏}

pullfrog

✅ No new issues found.

Reviewed changes

Added continue after each specialized tool output block — prevents fall-through to the unconditional function_call_output push for apply_patch_call_output, shell_call_output, and local_shell_call_output.
Fixed shell tool name lookup — changed "OpenAiFunctionShell" to "OpenAiShell" on line 785, which was a latent bug where the shellTool variable was never populated.
Added three mocked-HTTP regression tests — each test verifies that a specialized tool result produces exactly the expected output type and no duplicate generic function_call_output. Covers apply_patch, shell, and local_shell.

All prior review feedback (missing implementation fix, missing tests for shell and local_shell) is addressed. All three new tests pass.

^{｜ View workflow run ｜ Using DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏}

github-actions · 2026-08-03T21:55:18Z

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name	Current Size	Previous Size	Difference
`basic.ts`	7.06 KB	7.06 KB	0.00 KB (0.00%)
`batching.ts`	9.86 KB	9.86 KB	0.00 KB (0.00%)
`brand.ts`	6.34 KB	6.34 KB	0.00 KB (0.00%)
`cache.ts`	10.62 KB	10.62 KB	0.00 KB (0.00%)
`config.ts`	20.60 KB	20.60 KB	0.00 KB (0.00%)
`differ.ts`	20.20 KB	20.20 KB	0.00 KB (0.00%)
`http-client.ts`	21.49 KB	21.49 KB	0.00 KB (0.00%)
`logger.ts`	10.76 KB	10.76 KB	0.00 KB (0.00%)
`metric.ts`	8.99 KB	8.99 KB	0.00 KB (0.00%)
`optic.ts`	7.18 KB	7.18 KB	0.00 KB (0.00%)
`pubsub.ts`	14.90 KB	14.90 KB	0.00 KB (0.00%)
`queue.ts`	11.58 KB	11.58 KB	0.00 KB (0.00%)
`schedule.ts`	10.74 KB	10.74 KB	0.00 KB (0.00%)
`schema-class.ts`	19.14 KB	19.14 KB	0.00 KB (0.00%)
`schema-fromJsonSchemaDocument.ts`	28.96 KB	28.96 KB	0.00 KB (0.00%)
`schema-representation-roundtrip.ts`	25.29 KB	25.29 KB	0.00 KB (0.00%)
`schema-string-transformation.ts`	13.30 KB	13.30 KB	0.00 KB (0.00%)
`schema-string.ts`	10.94 KB	10.94 KB	0.00 KB (0.00%)
`schema-template-literal.ts`	15.17 KB	15.17 KB	0.00 KB (0.00%)
`schema-toArbitraryLazy.ts`	21.94 KB	21.94 KB	0.00 KB (0.00%)
`schema-toCodeDocument.ts`	24.34 KB	24.34 KB	0.00 KB (0.00%)
`schema-toCodecJson.ts`	19.18 KB	19.18 KB	0.00 KB (0.00%)
`schema-toEquivalence.ts`	19.01 KB	19.01 KB	0.00 KB (0.00%)
`schema-toFormatter.ts`	18.87 KB	18.87 KB	0.00 KB (0.00%)
`schema-toJsonSchemaDocument.ts`	22.60 KB	22.60 KB	0.00 KB (0.00%)
`schema-toRepresentation.ts`	19.52 KB	19.52 KB	0.00 KB (0.00%)
`schema.ts`	18.41 KB	18.41 KB	0.00 KB (0.00%)
`stm.ts`	12.54 KB	12.54 KB	0.00 KB (0.00%)
`stream.ts`	9.79 KB	9.80 KB	-0.01 KB (-0.08%)

Emit specialized OpenAI tool results only once

95459f6

fubhy added the audit Findings originating from the Effect runtime correctness audit label Aug 3, 2026

github-project-automation Bot added this to PR Backlog Aug 3, 2026

github-project-automation Bot moved this to Discussion Ongoing in PR Backlog Aug 3, 2026

pullfrog Bot requested changes Aug 3, 2026

View reviewed changes

github-project-automation Bot moved this from Discussion Ongoing to Waiting on Author in PR Backlog Aug 3, 2026

Fix specialized OpenAI tool outputs

5655ab2

pullfrog Bot approved these changes Aug 3, 2026

View reviewed changes

Fix OpenAI tool test constructors

8522a9f

tim-smart enabled auto-merge (squash) August 3, 2026 21:42

tim-smart merged commit 4686265 into main Aug 3, 2026
16 checks passed

tim-smart deleted the audit/repro-split-adapters-ai-9-openai-duplicate-specialized-tool-output branch August 3, 2026 21:53

github-project-automation Bot moved this from Waiting on Author to Done in PR Backlog Aug 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Emit specialized OpenAI tool results only once - #6921

Emit specialized OpenAI tool results only once#6921
tim-smart merged 3 commits into
mainfrom
audit/repro-split-adapters-ai-9-openai-duplicate-specialized-tool-output

fubhy commented Aug 3, 2026 •

edited by tim-smart

Loading

Uh oh!

changeset-bot Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

pullfrog Bot left a comment

Uh oh!

pullfrog Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

fubhy commented Aug 3, 2026 • edited by tim-smart Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Specialized OpenAI tool results are emitted twice

What happens

Why it happens

Expected behavior

Relevant implementation

Reproduction

Implementation handoff

Audit provenance

Uh oh!

changeset-bot Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pullfrog Bot left a comment

Choose a reason for hiding this comment

⚠️ Missing implementation fix

⚠️ Missing tests for shell and local_shell

Uh oh!

pullfrog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Aug 3, 2026

Bundle Size Analysis

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fubhy commented Aug 3, 2026 •

edited by tim-smart

Loading

changeset-bot Bot commented Aug 3, 2026 •

edited

Loading

⚠️ Missing tests for `shell` and `local_shell`