Skip to content

fix(llm): include expected and received keys in tool schema error message#30224

Open
nikhilkulkarni1755 wants to merge 1 commit into
anomalyco:devfrom
nikhilkulkarni1755:fix/tool-schema-error-message
Open

fix(llm): include expected and received keys in tool schema error message#30224
nikhilkulkarni1755 wants to merge 1 commit into
anomalyco:devfrom
nikhilkulkarni1755:fix/tool-schema-error-message

Conversation

@nikhilkulkarni1755
Copy link
Copy Markdown

@nikhilkulkarni1755 nikhilkulkarni1755 commented Jun 1, 2026

Issue for this PR

Closes #29142
Related to #18131 (original report, March 2026)

Type of change

  • Bug fix

What does this PR do?

When a local model sends wrong argument keys to a tool — e.g. fileContent instead of content for the write tool — the error fed back only says Invalid tool input: Missing key at ["content"]. The model has no idea what keys to use, so it retries identically and hits the doom loop.

This fixes the error message to include expected keys and received keys, so the model can self-correct on the next turn. The error is built in tool-runtime.ts (the dispatch layer), not individual tool definitions — that's where schema validation happens across all tools.

One implementation detail: Effect's JSON schema generator wraps multi-field structs in $defs/$ref (see tool.ts line 183), so reading inputSchema.properties directly is empty for tools with more than one parameter. The helper resolves both shapes.

Complementary to #29943 (field reorder, still open) and #30091 (settle pending parts, still open) — those prevent the mismatch and clean up DB state; this fixes what the model sees when it goes wrong anyway.

How did you verify your code works?

Ran bun test (full llm package) inside an Orbstack Ubuntu 24.04 container (Bun 1.3.14): 260 pass, 23 skip, 0 fail.

Added 3 tests covering the failure modes from the bug reports:

  • Wrong key names on write tool schema (fileContent/path vs content/filePath)
  • Missing required key on edit tool schema (token-budget truncation case)
  • Wrong key names on a single-field tool

test-results

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

The following comment was made by an LLM, it may be inaccurate:

Based on my search, the only potentially related PR is:

However, these are complementary fixes addressing different aspects of the same issue, not duplicates of PR #30224. The current PR specifically fixes the error message display to help models self-correct, while those PRs prevent/clean up the mismatch scenario.

No duplicate PRs found

…sage

When a local model sends wrong argument keys to a tool the error fed back
only said "Invalid tool input: Missing key at ["content"]". The model has
no signal about what keys to use, so it retries identically and hits the
doom loop.

Adds expected and received keys to the error so the model can self-correct
on the next turn. Also resolves Effect's $defs/$ref JSON schema shape for
multi-field structs so the expected keys are extracted correctly.

Fixes anomalyco#29142
@nikhilkulkarni1755 nikhilkulkarni1755 force-pushed the fix/tool-schema-error-message branch from e0f9b8d to 797837f Compare June 5, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI-compatible models can call write/edit with invalid schema arguments

1 participant