Skip to content

feat: add Cloudflare Workers AI and AI Gateway as model providers#3389

Merged
dgageot merged 1 commit into
mainfrom
feat/3354-3355-cloudflare-providers
Jul 1, 2026
Merged

feat: add Cloudflare Workers AI and AI Gateway as model providers#3389
dgageot merged 1 commit into
mainfrom
feat/3354-3355-cloudflare-providers

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

Adds two OpenAI-compatible provider aliases for Cloudflare, closing #3355 (Workers AI) and #3354 (AI Gateway). They share the CLOUDFLARE_* environment variables and are handled together.

Both endpoints are account/gateway scoped, so a fully static alias is not possible. The chosen approach is a templated base URL: the alias BaseURL carries ${CLOUDFLARE_ACCOUNT_ID} (and ${CLOUDFLARE_GATEWAY_ID} for the gateway), resolved from the environment at provider-build time via the existing ${env.X} expansion path (issue #2261). No new resolution machinery is introduced.

Provider Alias Base URL (templated) Required env
Workers AI cloudflare-workers-ai https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/ai/v1 CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID
AI Gateway cloudflare-ai-gateway https://gateway.ai.cloudflare.com/v1/${CLOUDFLARE_ACCOUNT_ID}/${CLOUDFLARE_GATEWAY_ID}/compat CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_GATEWAY_ID

Definition of Done

Expectation (both issues) Status Where
Usable as a provider (alias with templated base URL) yes pkg/model/provider/aliases.go
Unit / config test covering the setup yes see Testing below
Example YAML config yes examples/cloudflare-workers-ai.yaml, examples/cloudflare-ai-gateway.yaml
Docs updated (required env vars) yes docs/providers/cloudflare-*, overview, concepts, configuration, nav
JSON schema entry yes agent-schema.json

Design decisions

  • Templated base URL over custom-provider only. Reuses the ${env.X} expansion already applied to model/base_url, so a bare provider: cloudflare-workers-ai resolves the endpoint from the environment. A missing variable produces a clear error instead of dialing a malformed URL.
  • Preflight and picker made env-aware. The config preflight (pkg/config/gather.go) now also requires the env vars referenced by a templated alias base URL. The runtime model picker (pkg/runtime/model_switcher.go) only advertises such an alias once all of them are set, not on the token alone, so it never surfaces catalog models that cannot be selected (the embedded models.dev snapshot does list Cloudflare models).
  • Not wired into auto-selection. Cloudflare needs a conjunction of variables (token plus account, plus gateway), which the disjunctive "any key set" detection model in pkg/config/auto.go cannot express. Cloudflare stays an explicitly configured provider. Auto-selection is not part of either issue's Definition of Done.
  • System-message coalescing. Both are added to openModelHostProviders: Workers AI serves open-weight models directly, and the gateway can front them.

AI Gateway authentication (residual design point)

Verified against the Cloudflare docs. The alias sends CLOUDFLARE_API_TOKEN in the standard Authorization: Bearer header, so it works with a gateway that has authentication disabled (the default), routing to Workers AI models. A gateway with authentication enabled (unified billing) instead expects the cf-aig-authorization header; the alias does not send it, and provider_opts.http_headers values are not environment-expanded, so that case is not supported out of the box. This is documented on the provider page with a custom-provider fallback. Adding a first-class, env-expanded gateway-auth header can be a follow-up if unified billing is wanted; the current change does not preclude it.

Testing

Area Test
Alias definitions TestCloudflareAliases (pkg/model/provider/aliases_test.go)
Templated URL resolution and missing-env errors pkg/model/provider/cloudflare_alias_test.go
End-to-end (auth, routing, system-message merge) rows in pkg/model/provider/openai_alias_providers_test.go (hermetic, plus opt-in live via CLOUDFLARE_*)
Preflight env gathering pkg/config/gather_alias_env_test.go
Picker availability requires all env vars TestGetAvailableProviders_TemplatedAlias (pkg/runtime/model_switcher_test.go)

go build ./..., go vet, and golangci-lint (0 issues) pass; tests for the affected packages (pkg/config, pkg/model/provider, pkg/runtime) pass.

Not covered

No live request was executed against the Cloudflare API in this change. The opt-in TestOpenAIAliasProvider_LiveAPI runs a real request when CLOUDFLARE_API_TOKEN (and the account/gateway ids) are present, and stays skipped otherwise.

Add two OpenAI-compatible aliases whose base URLs are account/gateway scoped
and therefore templated with CLOUDFLARE_* env vars, resolved at provider-build
time via the existing ${env.X} expansion path (issue #2261):

- cloudflare-workers-ai: .../accounts/${CLOUDFLARE_ACCOUNT_ID}/ai/v1
- cloudflare-ai-gateway: .../v1/${CLOUDFLARE_ACCOUNT_ID}/${CLOUDFLARE_GATEWAY_ID}/compat

Both authenticate with CLOUDFLARE_API_TOKEN and coalesce consecutive system
messages, like the other open-model hosts. The config preflight now also
requires the env vars referenced by a templated alias base URL, and the runtime
model picker only advertises such an alias once all of them are set (not on the
token alone), so it never surfaces catalog models that cannot be selected.

Cloudflare is deliberately not wired into auto-selection: its conjunctive env
requirement (token plus account, plus gateway) cannot be expressed by the
disjunctive "any key set" detection model, so it stays an explicitly configured
provider.

Verified against the Cloudflare docs. The AI Gateway alias sends the token in
the Authorization header, so it works with an unauthenticated gateway routing to
Workers AI models. An authenticated gateway (unified billing) expects the
cf-aig-authorization header and is not supported out of the box; this is
documented with a custom-provider fallback.

Wires the change into the JSON schema, docs (two provider pages, nav, overview,
concepts, configuration), example configs, and tests: templated URL resolution,
missing-env errors, the shared alias end-to-end harness, the preflight env
gathering, and picker availability.

Closes #3354
Closes #3355
@Sayt-0 Sayt-0 requested a review from a team as a code owner July 1, 2026 16:44
@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 1, 2026
@dgageot dgageot merged commit b2aadb1 into main Jul 1, 2026
13 checks passed
@Sayt-0 Sayt-0 deleted the feat/3354-3355-cloudflare-providers branch July 1, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants