Skip to content

feat: provider-executed tools #6815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 80 commits into from
Jun 23, 2025
Merged

feat: provider-executed tools #6815

merged 80 commits into from
Jun 23, 2025

Conversation

lgrammel
Copy link
Collaborator

@lgrammel lgrammel commented Jun 22, 2025

Background

Many providers offer tools that are executed on their server and provide tool results, e.g. search and code execution tools. Omitting those outputs from the LLM history affects future calls (e.g. the LLM needs to know that it used a web search tool).

Summary

Introduces provider-executed tool calls, results and errors at all levels (language model, ai core, ui). Adds anthropic websearch support for full feature set.

Verification

End-to-end UI test: sources with anthropic websearch tool. render tool status and input.

Tasks

  • spec updates
  • send tool result from anthropic web search tool (stream)
  • send tool error from anthropic web search tool (stream)
  • send tool result from anthropic web search tool (generate)
  • send tool error from anthropic web search tool (generate)
  • generateText support for result with test
  • generateText exclude provider side tool calls and results from roundtrip mgmt
  • generateText support for error with test
  • streamText support for result with test
  • streamText support for error with test
  • streamText exclude provider side tool calls and results from roundtrip mgmt
  • add providerExecuted flag to tool calls, results, and errors
  • UI extensions
  • UI message mapping
  • response message mapping
  • simulate streaming middleware
  • server side tool calls and results for openai file search
  • server side tool calls and results for openai web search
  • rm encrypted content from provider metadata
  • add roundtrip generateText with server side tool example
  • provider metadata on tool calls and results
  • tool result error should have json value
  • ui input streaming provider executed

Future Work

  • implement google code execution tool
  • implement full support for openai file and web search

Related Issues

Builds on #6807

@lgrammel lgrammel changed the title branch feat: server-side tool results Jun 22, 2025
@lgrammel lgrammel marked this pull request as ready for review June 23, 2025 18:04
@lgrammel lgrammel merged commit e7d2ce3 into v5 Jun 23, 2025
7 checks passed
@lgrammel lgrammel deleted the lg/gYunBGuM branch June 23, 2025 18:11
lgrammel added a commit that referenced this pull request Jun 24, 2025
…ation (#6838)

## Background

Tool names for provider defined tools were removed in #6815. They are
needed for future validation of the tool setup.

## Summary

Add provider defined tool names back.

## Future Work

Validate that provider defined tools are used with their correct name in
`streamText` and `generateText` requests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants