Skip to content

Preserve code fences around standalone tool JSON and add marker-output guards#132

Merged
CJackHwang merged 1 commit intodevfrom
codex/toolcallhistory-6t7271
Mar 21, 2026
Merged

Preserve code fences around standalone tool JSON and add marker-output guards#132
CJackHwang merged 1 commit intodevfrom
codex/toolcallhistory-6t7271

Conversation

@CJackHwang
Copy link
Owner

Motivation

  • Avoid consuming legitimate closed code fences when a standalone tool JSON payload follows and prevent leaking tool-history markers into assistant output.

Description

  • Implemented fence-aware parsing by adding trimWrappingJSONFence and openFenceStartBefore to internal/adapter/openai/tool_sieve_core.go and the corresponding trimWrappingJSONFence in internal/js/helpers/stream-tool-sieve/sieve.js to only strip opening fences that truly wrap standalone tool JSON.
  • Updated findToolSegmentStart and consumeToolCapture to use the new fence helpers and to preserve trailing prose/closing fences when appropriate.
  • Added guard instructions to prompts to prevent models from emitting [TOOL_CALL_HISTORY] or [TOOL_RESULT_HISTORY] markers in internal/adapter/claude/handler_utils.go and internal/adapter/openai/handler_toolcall_format.go and updated related tests to expect the new instructions.
  • Added and adjusted tests to verify behavior, including TestHandleStreamStandaloneToolCallAfterClosedFenceKeepsFence in Go and the sieve preserves closed fence before standalone tool payload test in Node, plus minor updates to existing prompt/build tests.

Testing

  • Ran Go unit tests for the modified packages including internal/adapter/openai and internal/adapter/claude, and the new/updated tests (e.g. TestHandleStreamStandaloneToolCallAfterClosedFenceKeepsFence) passed.
  • Ran Node unit tests for the stream tool sieve (tests/node/stream-tool-sieve.test.js), including the new fence-preservation test, and they passed.
  • Stream handling assertions confirming finish_reason=tool_calls and no leakage of raw tool_calls or stray code fences were validated and succeeded.

Codex Task

@vercel
Copy link

vercel bot commented Mar 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ds2api Ready Ready Preview, Comment Mar 21, 2026 9:39am

@CJackHwang CJackHwang merged commit 67787d9 into dev Mar 21, 2026
3 checks passed
@CJackHwang CJackHwang deleted the codex/toolcallhistory-6t7271 branch March 21, 2026 09:44
CJackHwang added a commit that referenced this pull request Mar 21, 2026
Merge pull request #132 from CJackHwang/codex/toolcallhistory-6t7271

Preserve code fences around standalone tool JSON and add marker-output guards
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] 进行toolcall时保留原始json

1 participant