Skip to content

fix: Repair truncated JSON tool inputs in LLM session#24289

Open
menardorama wants to merge 2 commits intoanomalyco:devfrom
menardorama:tool_call_fix
Open

fix: Repair truncated JSON tool inputs in LLM session#24289
menardorama wants to merge 2 commits intoanomalyco:devfrom
menardorama:tool_call_fix

Conversation

@menardorama
Copy link
Copy Markdown

@menardorama menardorama commented Apr 25, 2026

Issue for this PR

Fixes #24177 #20650 #20786

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Use jsonrepair library to fix truncated/malformed JSON in tool call arguments from LLMs like Kimi k2.6 and Qwen 3.5 running on vLLM.

Changes

  • Replaced ~90 lines of custom JSON repair logic with the jsonrepair library
  • Much more robust and handles complex truncation cases
  • Reduced code footprint by ~70%
  • Better maintainability

How did you verify your code works?

  • Tested jsonrepair library with actual truncated JSON examples
  • Local build successful
  • Kimi k2.6 and Qwen 3.5 no longer throw JSON Parse errors

Screenshots / recordings

Before:

Thinking: The user wants me to review their project. I need to explore the codebase to understand what it is, what technologies it uses, and provide a review. Let me start by exploring the directory structure and key files.
I'll use the task tool to explore the codebase thoroughly.
⚙ invalid [tool=task, error=Invalid input for tool task: JSON parsing failed: Text: {"description": "Explore project structure", "prompt": "Explore the codebase at /Users/tmenard/work/dsi-docs-fumadocs to understand:\n1. What kind of project this is (framework, language, purpose)\n2. The directory structure and key files\n3. Dependencies and tech stack\n4. Any obvious issues or areas for improvement\n5. Quality of documentation, code organization, etc.\n\nProvide a comprehensive summary of what you find.", "subagent_type": "explore.
Error message: JSON Parse error: Unterminated string]
Thinking: The user wants me to review the project in the current working directory. Let me explore the codebase structure first.
⚙ invalid [tool=bash, error=Invalid input for tool bash: JSON parsing failed: Text: {"command": "ls -la /Users/tmenard/work/dsi-docs-fumadocs", "description": "List files in project.
Error message: JSON Parse error: Unterminated string]

After:

Thinking: The user wants me to review a project. I need to first understand what project we're in by looking at the directory structure and key files. Let me start by exploring the workspace to understand what kind of project this
✱ Glob "*" (100 matches)
✱ Glob "**/package.json" (100 matches)
✱ Glob "**/tsconfig.json" (6 matches)
→ Read package.json 
→ Read README 
File not found: /Users/tmenard/work/dsi-docs-fumadocs/README
Did you mean one of these?
/Users/tmenard/work/dsi-docs-fumadocs/README.md

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions Bot added needs:compliance This means the issue will auto-close after 2 hours. needs:title labels Apr 25, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Hey! Your PR title Repair truncated JSON tool inputs in LLM session doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@github-actions
Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Found a potentially related PR:

PR #23067: fix(opencode): repair malformed JSON in tool call arguments
#23067

This PR appears to address a similar issue of repairing malformed JSON in tool call arguments, which is closely related to the current PR's focus on repairing truncated JSON tool inputs.

@menardorama menardorama changed the title Repair truncated JSON tool inputs in LLM session fix: Repair truncated JSON tool inputs in LLM session Apr 25, 2026
@github-actions github-actions Bot removed needs:title needs:compliance This means the issue will auto-close after 2 hours. labels Apr 25, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@menardorama
Copy link
Copy Markdown
Author

feel free to request more details if needed; we are totally stuck with our local models due to this bug.
Thanks

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@menardorama
Copy link
Copy Markdown
Author

@rekram1-node Hi do you have time to review this PR ?

It's been 2 days that users on my side are running opencode with this patch with so far no issue.

This solve tool calling errors for kimi k2.6 and qwen 3.5 running on vllm (plain huggingface version).

Thanks a lot

Replace ~90 lines of custom JSON repair logic with jsonrepair library.
This handles unterminated strings, missing braces/brackets, trailing
commas, and other common truncation issues from LLMs like Kimi k2.6
and Qwen 3.5 running on vLLM.

Refs: anomalyco#24177, anomalyco#20650, anomalyco#20786
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool calls fail with JSON Parse error when description ends with a dot

1 participant