Skip to content

Support deferred loading of tools and discovery via tool search tool #3590

@DouweM

Description

@DouweM

Description

Inspired by Anthropic's https://www.anthropic.com/engineering/advanced-tool-use and https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool, but implemented in Pydantic AI so it works for all providers. From those docs:

Prompt caching note: Tool Search Tool doesn't break prompt caching because deferred tools are excluded from the initial prompt entirely. They're only added to context after Claude searches for them, so your system prompt and core tool definitions remain cacheable.

Unfortunately this wouldn't apply to our native implementation as the only way we have of passing in tool definitions is through the official list at the start of the context, that we'd vary at each step of the agent run based on what's already been discovered. It looks like Anthropic has a special method of injecting new tool definitions at a later point of the conversation/context,

Related notes from #3550 (comment):

But on the other hand, defer_loading seems like a reasonable not-Anthropic-specific name, and I could see how we could implement support for this in Pydantic AI using a new flag that will insert our own tool search tool.

So I'm OK with this field, but I'd want it to automatically add the ToolSearchTool when used. Similar to how specifying output_type=BinaryImage on OpenAIResponsesModel automatically adds the ImageGenerationTool, and how using strict=True on a ToolDefinition automatically adds the structured-outputs-2025-11-13 Anthropic beta header.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions