feat (provider/neon): add Neon AI Gateway provider#15997
Closed
andrelandgraf wants to merge 5 commits into
Closed
Conversation
Add `@ai-sdk/neon`, a provider for the branch-scoped Neon AI Gateway. It targets the gateway's unified, OpenAI-compatible MLflow endpoint, exposing the full `databricks-*` foundation model catalog (Anthropic, OpenAI, Google, Meta, and more) through a single language model. Configure with `NEON_AI_GATEWAY_BASE_URL` (the branch host) and `NEON_AI_GATEWAY_TOKEN` (the platform token), or pass `baseURL`/`apiKey` to `createNeon`. Includes provider + chat-model unit tests, an example, and docs.
Verified the adapter against the live gateway across Anthropic, OpenAI, Google, and Meta models for text, streaming, tools, structured output, and vision. - Strip the JSON Schema `$schema` marker from tool parameters and structured output schemas via a request transform. Gemini's backend rejects unknown schema fields, which previously broke tool calling and object generation for Gemini; other backends ignore its absence. - Enable `supportsStructuredOutputs` so `generateObject` uses `response_format: json_schema` instead of the `json_object` fallback (which required the prompt to contain the word "json" and failed on several backends). - Document supported capabilities, per-backend sampling-parameter constraints, the streaming-structured-output limitation for Databricks-hosted Anthropic models, and that image generation / embeddings are not offered by the gateway.
The Neon AI Gateway proxies to heterogeneous backends that each accept a different subset of OpenAI-style parameters; previously, sending a parameter a backend didn't support failed the whole request with a hard 400. Add prefix-based model-family capability detection (getNeonModelCapabilities) and drop parameters a family is known to reject, surfacing an AI SDK warning instead of erroring — mirroring how @ai-sdk/openai handles reasoning models: - Anthropic: drops frequencyPenalty/presencePenalty/seed, drops topP when temperature is also set, drops reasoningEffort. - OpenAI gpt-5* reasoning: drops penalties and stop; drops temperature/topP for gpt-5 / gpt-5-mini / gpt-5-nano (gpt-5.1+ keep them). - Meta (Llama): drops frequencyPenalty/presencePenalty/seed. - Google (Gemini): drops reasoningEffort. - Unknown/untested models are passed through unchanged. Verified live across Anthropic, OpenAI, Google, and Meta models: the full capability matrix (text, system, sampling, penalties, stop, seed, streaming, tools, generateObject, reasoningEffort, vision) now passes, with dropped parameters reported as warnings.
…llback Some Neon AI Gateway models (e.g. Codex) are only served via provider-native endpoints, so route by model family instead of always using the unified endpoint: - Anthropic (claude) -> native Messages API (unlocks streaming structured output; disables `eager_input_streaming`, which the gateway rejects). - OpenAI (gpt-*, codex) -> native Responses API (unlocks Codex, which is not available on the chat endpoint; sets `forceReasoning` for gpt-5 models so the Responses model's reasoning handling works despite the `databricks-` prefix). - Everything else (Gemini, Llama, Qwen, gpt-oss, ...) -> unified MLflow endpoint. Gemini stays here because its native gateway endpoint does not support streaming. Routing is transparent (same base URL + token). Reuses the official @ai-sdk/anthropic and @ai-sdk/openai model classes rather than reimplementing them. MLflow-routed models keep the prefix-aware capability handling. Verified live: text, system, sampling, penalties, stop, seed, streaming, tools (single + multi-step + streaming), generateObject, streamObject, reasoning, and vision pass across Anthropic, OpenAI, Codex, Google, and Meta models.
Image generation works today via the OpenAI Responses `image_generation` tool when using `streamText` (the image returns as a `tool-result` part). Streaming avoids the gateway's non-streaming response-size cap and read timeout, so only the single-shot `generateText` path is constrained. Clarify the docs accordingly.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
@ai-sdk/neon, a provider for the branch-scoped Neon AI Gateway.Each Neon project branch gets its own gateway host, and a platform token authorizes requests for that branch. The provider routes each model to the best gateway endpoint based on its id, so a single
neon('databricks-...')call (same base URL + token) reaches the fulldatabricks-*catalog with the best available feature support.Routing
databricks-claude-*)databricks-gpt-*,*-codex)Routing reuses the official
@ai-sdk/anthropicand@ai-sdk/openaimodel classes (no reimplementation), with thin Neon subclasses that handle gateway quirks:forceReasoningforgpt-5*so reasoning handling works despite the requireddatabricks-prefix.eager_input_streaming(rejected by the gateway) so streaming tool calls work.Configuration
NEON_AI_GATEWAY_BASE_URL(branch host root) +NEON_AI_GATEWAY_TOKEN(platform token), both from the Console's AI Gateway tab; or passbaseURL/apiKeytocreateNeon.Verified live (Anthropic, OpenAI, Codex, Google, Meta)
generateText,streamText, system prompts, multi-turn, tool calling (single + multi-step + streaming),generateObject,streamObject, reasoning, and image (vision) input.For MLflow-routed models, the provider detects the family and drops parameters a backend rejects (e.g. penalties/
seedfor Llama,reasoningEffortfor Gemini) with an AI SDK warning instead of a hard error.Known gateway limitations (not adapter bugs; documented)
generateImage) and embeddings (embed/embedMany) have no gateway endpoint →NoSuchModelError. (Image gen exists only as the OpenAI Responsesimage_generationtool and is constrained by the gateway's response-size cap.)gpt-oss-*returns a non-standard response shape on the unified endpoint.databricks-gpt-5-mini+ multi-step tools intermittently returns gateway502s (upstream); Codex and other models are unaffected.Test plan
pnpm --filter @ai-sdk/neon test(node + edge),type-check,build