fix(ai-sdk): keep output validation enabled for tool procedures - #1791
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
More templates
@orpc/ai-sdk
@orpc/arktype
@orpc/bun
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/hibernation
@orpc/json-schema
@orpc/nest
@orpc/next
@orpc/openapi
@orpc/opentelemetry
@orpc/pinia-colada
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/swr
@orpc/tanstack-query
@orpc/trpc
@orpc/valibot
@orpc/zod
commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes: Reviewed the full diff of commit a5b928a (the repository's only commit for this PR) in packages/ai-sdk/src/tool.ts and packages/ai-sdk/src/tool.test.ts.
- Output validation re-enabled —
createToolFactorynow disables onlydisableInputValidation;disableOutputValidationstays on, so tool procedures validate.output()schemas (including transforms/defaults) that the AI SDK never validates. - Iterator
executesimplified — replaced theisAsyncIteratorObjectfallback withyield* await call(...), relying on the (now enabled)asyncIteratorObjectvalidation to reject non-iterators and validate each yielded event. - New premise-guard test — a
generateText+MockLanguageModelV4test proves the AI SDK passes schema-violatingexecuteoutput through untouched, and would fail if a futureairelease adds output validation. - Reworked tests — cover input validation being skipped, invalid output rejection, per-event stream validation, and non-iterator rejection.
I traced the behavior through procedure-client.ts (validateOutput/validateInput) and the asyncIteratorObject schema in packages/contract/src/schema-built-in.ts: validate() returns { issues } for non-iterators (→ "Output validation failed") or a wrapped iterator that validates each event (→ "AsyncIteratorObject validation failed"). The new tests map onto exactly these code paths and each genuinely fails under the prior validation-disabled behavior. All 20 tests in tool.test.ts pass and lint is clean. The single caveat—the generateText test uses current AI SDK v4 shapes and is version-fragile test infra—is acknowledged and not a code concern.
This is a correct and well-tested fix that closes a real validation gap.
openrouter/~deepseek/deepseek-v4-flash-latest (free via Pullfrog for OSS) | 𝕏

createToolFactorydisabled both input and output validation on the assumption that the AI SDK re-validates against the tool schemas. That is only true for input: the AI SDK treatsoutputSchemaas type metadata and never validates the valueexecutereturns against it (confirmed in vercel/ai#10222, docs corrected in vercel/ai#11016). Procedures run as tools therefore skipped their.output()schemas entirely, including transforms and defaults.Fixes
asyncIteratorObjectoutputs validate every yielded event, and a handler returning a non-iterator now errors instead of being yielded once as the final result (the old fallback branch became unreachable and is removed).Testing
generateTexttest with a mock model proves the AI SDK returns schema-violatingexecuteoutput untouched, so it will start failing if a futureairelease adds output validation and makes ours redundant.