Support useChat Without an API Route (Client-side usage)

[The-Best-Codes]

Feature Description

Right now, the Vercel AI SDK's useChat hook is good for managing chat state, but it requires an API route (e.g., app/api/chat/route.ts). So even in simple projects or static sites, a backend is necessary just to proxy requests.
I would like to be able to use the useChat hook right on the client with a user-provided API key, with no need for an API route.

Use Cases

I can think of a lot of use cases:

• Static site generators (Next.js static export, Astro, etc.)
• No-API web-to-desktop tools, e.g., Tauri
• Browser extensions
• Vite projects

Additional context

No response

[The-Best-Codes]

Feel free to assign me!
I've got a solution I'm using locally (for the React useChat hook). Let me know how you guys would implement this. 😊

[chewong]

For anyone who is looking for a pure-frontend hacky solution (I hosted a Llama 3 model with vLLM at port 5000)

const model = createOpenAICompatible({
  name: "model",
  baseURL: "http://localhost:5000/v1/"
})
const customFetch = async (url, options) => {
  const m = JSON.parse(options.body) as any;
  console.log(m)
  const result = await streamText({
    model: model.chatModel('meta-llama/Llama-3.1-8B-Instruct'),
    messages: m.messages,
    abortSignal: options.signal,
  });
  return result.toDataStreamResponse()
}

const { messages, input, handleInputChange, handleSubmit } = useChat({
  fetch: customFetch,
});

[The-Best-Codes]

@chewong This looks great, I will try it 👍

[xhiroga]

Thank you for the issue and PR. This feature is very necessary.
When creating application extensions, for example, we often need to attach a web server to an existing process.
In such cases, in order to use the AI SDK, it must be available from the hosted application in a statically exported state.

[lgrammel]

ai sdk 5 supports custom chat transports, which enables client only usage

[The-Best-Codes]

Awesome!! ❤
Should my PR be closed too?
#5207