Tools with Generate Object

[arnau01]

Feature Description

Generating structured data is very helpful, would be great if tool calling was supported.

Use Case

Providing a output schema and having access to external tools

Additional context

No response

[lgrammel]

generateObject uses forced tool calling under the hood for some providers, so this might not be possible.

[arnau01]

Makes sense, my current workaround is a simple chain, first generateText then streamObject which allows for tool usage and also controlling the schema.

[stephenkalnoske]

Makes sense, my current workaround is a simple chain, first generateText then streamObject which allows for tool usage and also controlling the schema.

Can you give an example of this?

[mi3lix9]

You could throw an error with providers which use tool calling and enable it for natively support json mode

[michaeloliverx]

How can I use tool calling along with generateObject?

My structured output requires some lookups to generate its data, how can I work around this for now?

[lgrammel]

@michaeloliverx you can just do the lookups before calling generateObject and add the result to the prompt.

[michaeloliverx]

The queries to lookup need to be inferred by the LLM itself then I would like a chain of thought structured output back. AFAIK structured-outputs and tool calling is supported in "pure" openai SDK usage?

https://platform.openai.com/docs/guides/structured-outputs/examples?lang=node.js

[lgrammel]

Not quite the same but you can use the "Agents with Structured Answers" approach: https://sdk.vercel.ai/docs/ai-sdk-core/agents#structured-answers