Validate common AI tool-call arguments to enable pydantic-ai retries - #70096
Conversation
Malformed model calls currently reach tool implementations and can fail the task with KeyError or TypeError instead of giving the model a bounded retry.
SameerMesiah97
left a comment
SameerMesiah97
left a comment
There was a problem hiding this comment.
I think the title is a bit misleading. It appears that the malformed payload is not retried, but rather rejected during validation, allowing Pydantic AI to regenerate a new tool call. "Retry malformed tool calls" initially reads as though the original invalid payload is executed again, whereas the retry is actually another model generation. It might be worth clarifying this in the title and PR description.
This looks fine to me logically (and the tests are solid) but this will need someone who is more familiar with this provider to deeply vet the validation logic. I see a lot of small changes scattered throughout the diff that I am not sure of how to ascertain the impact of.
CI needs to be triggered too.
zozo123
changed the title
|
Thanks @SameerMesiah97 — good catch on the title. Renamed to "Validate common AI tool-call arguments to enable pydantic-ai retries" and added a line to the description making explicit that the malformed payload is rejected at validation and pydantic-ai regenerates a fresh tool call (the original is never re-executed). Also replied inline on the nested-object question. |
Unify object-fragment handling so build_args_validator and the "object" branch of _fragment_to_core_schema share one builder. Nested objects with declared properties are now validated recursively, matching how arrays already recurse into their items. Objects with no "properties" key (e.g. from dict[K, V]) still accept any dict, so untyped payloads are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
potiuk
added
the
ready for maintainer review
… budget The strict argument validators would otherwise regress the bridge path from a bounded retry to an uncaught ValidationError, aborting the run instead of letting the model correct its call.
256be17 to
4fc0eaf
Compare
Catch ValidationError only around arg validation on the LangChain bridge, and forbid undeclared args for fixed signatures so mistyped fields become bounded retries instead of silent partial calls. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
5 participants
Malformed model-generated arguments currently bypass validation in the Common AI toolsets. Missing or mistyped arguments can therefore reach tool implementations and fail with
KeyErrororTypeErrorinstead of producing the bounded retry that pydantic-ai supports.Invalid payloads are now rejected at validation; pydantic-ai then regenerates a new tool call — the original malformed payload is never re-executed.
Concretely: if the model calls the
querytool but omits the requiredsqlargument, the call previously crashed the run withKeyError: 'sql'; it now raises aValidationErrorthat pydantic-ai turns into one bounded retry, and the model regenerates the call withsqlsupplied.This change derives each tool's argument validator from the same JSON Schema advertised to the model and applies it to
SQLToolset,DataFusionToolset, andHookToolset. Hook introspection keeps optional and union parameters accurate, leaves unsupported annotations permissive, rejects unexpected arguments for fixed signatures (so a mistyped field becomes a bounded retry, matching native pydantic-ai), and preserves additional arguments for methods that accept**kwargs.The LangChain bridge mirrors the same two-stage semantics:
ValidationErrorfrom arg validation is fed back as a retry;ValidationErrorraised insidecall_toolpropagates so a non-idempotent tool is not re-invoked.Validation:
forbidextras passedWas generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Fable 5) and Codex (GPT-5) following the guidelines