feat (provider/openai): support file search tool

[matthewmichel]

Extending the openai-tools to support the "file_search" tool call documented by OpenAI here: https://platform.openai.com/docs/guides/tools-file-search?lang=javascript

Note: originally set this up as "file_search_preview", but opted for "file_search" because OpenAI does not have this tool tagged as a preview feature and the tool name in the OpenAI docs is "file_search".

[lgrammel]

Nice! without parsing their special outputs how will this work in practice?

[matthewmichel]

When I tested it to see what would happen when adding in a vector_store_id it returned a response as expected through the "text" value with references to the vector store files (indicating to me that it worked to perform the file search).

The "text" value (returned by the new OpenAI Responses API) should work as expected when using the file_search tool. I'll work on adding some handlers for the new "annotations" value that is returned by the Responses API for better clarity.

[matthewmichel]

@lgrammel I should have checked before responding. The "annotations" property is already available on the OpenAIResponsesAssistantMessage. This should be enough to pass through those annotations through the SDK response.

export type OpenAIResponsesAssistantMessage = {
  role: 'assistant';
  content: Array<{ 
    type: 'output_text'; 
    text: string; 
    annotations?: Array<{
      type: 'file_citation';
      index: number;
      file_id: string;
      filename: string;
    } | {
      type: 'url_citation';
      start_index: number;
      end_index: number;
      url: string;
      title: string;
    }>;
  }>;
};

This matches the OpenAI docs for the file search tool: https://platform.openai.com/docs/guides/tools-file-search

EDIT: I didn't realize I actually made this change 🤦‍♂️ The update to this type has been pushed to this PR.

[matthewmichel]

Ok made some adjustments to ensure the annotations information is being returned properly through the sdk. Tested this locally and got a nice sources value when trying out with the file_search tool.

sources: [
    {
      sourceType: 'file',
      id: 'PSvUjMxv2hlJ4vuj',
      fileId: 'file-IhP983o4sPmYkgGOtVdureJO',
      filename: 'Artificial Intelligence (AI) Policy.txt'
    }
  ]

I've also updated the next-openai examples to account for either url or file source annotations.

[lgrammel]

this is a potentially critical change because it also needs to work for anthropic and google. i would need more time to evaluate, most likely want to do this myself.

[matthewmichel]

I see your PR for the computer use tool. Would you like me to adjust this PR to follow those same practices for this file_search tool?

[cosbgn]

Any chance we could get this release as experimental_alpha and then migrate to one which supports all providers? I would love to migrate a lot of Assistants to the new responses api but I'm blocked by the file_search limitation

[MwSpaceLLC]

Waiting for this file_search adapter,

A game changer for ALL 🙂‍↔️

[IKatsuba]

This PR will close #5188

[nsenthilkumar]

Is there any update on this @lgrammel ?

[shrix1]

Hey, @lgrammel any update ?

[kvnwolf]

@lgrammel sorry for the possible spam, but any plans to merge this soon? Any change needed to the PR?

[andresgutgon]

Looking forward to this be merged 💪

[ruidpm]

@lgrammel any updates on this?

[lagorsse]

seems like blocked, I really need this tool, I could use assistant to move this into openai could work for some of you, but it doesn't work for me as I am trying to use assistant ui cloud which doesn't support assistant.

[Jaikant]

@lgrammel do you need any help on this? It seems the responses api is not compatible with APIs from Anthropic/xAI especially these custom constructs like file_search, web_search in the responses API?

https://apimagic.ai/blog/switching-assistant-responses-api

We can either use the existing tools parameter available on the ai-sdk methods or add it to the providerOptions like below:

const providerOptions = {
  openai: {
    tools: [
      {
        type: 'file_search',
        vector_store_ids: ['vs_1234567890'], 
        max_num_results: 20,
      },
    ],
    tool_choice: { type: 'tool', tool_name: 'file_search' },
  },
};

[ssg-chris]

Any news?

[lgrammel]

This functionality is available in ai sdk 5 beta.