feat: add LiteLLM as AI gateway provider

[RheagalFire]

Summary

Adds LiteLLM as a provider, enabling access to 100+ LLM backends through a LiteLLM proxy server. Follows the vLLM provider pattern (OpenAI SDK with custom baseURL, dynamic model discovery via /v1/models).

Fixes #2859

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• Other: ___________

Testing

Model discovery:

Discovered models: [ 'litellm/anthropic/claude-sonnet-4-6' ]

Basic completion through LiteLLM proxy:

Content: 4
Model: claude-sonnet-4-6
Tokens: {"completion_tokens":5,"prompt_tokens":20,"total_tokens":25}

Streaming:

Hello!
Full content: Hello!
PASS: streaming works

Tool calling:

Tool call: {"arguments":"{\"city\": \"London\"}","name":"get_weather"}
PASS: tool calling works

All tests use OpenAI SDK with baseURL: http://localhost:4000/v1 pointing at a LiteLLM proxy, which is exactly what the provider does.

Deep-dive bugs found and fixed:

• attachments.ts: formatMessagesForProvider missing litellm (would hit exhaustive never check at runtime)
• blocks/utils.ts: litellm not excluded from API key requirement check (dynamic providers don't need pre-configured keys)
• providers/utils.ts: missing metadata, getBaseModelProviders exclusion, and getApiKey handler
• stores/providers: missing ProviderName type and store defaults

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		May 17, 2026 10:51pm

[cursor]

PR Summary

Medium Risk
Adds a new OpenAI-compatible provider with streaming and tool-calling support, touching core provider execution and model-discovery paths. Risk is moderate due to new network/config surface (LITELLM_* envs) and potential edge cases in tool-iteration/stream handling, but changes are largely additive and follow existing vLLM patterns.

Overview
Adds LiteLLM as a first-class provider, including a new /api/providers/litellm/models route that discovers models via the proxy’s /v1/models, applies provider/model blacklists, and caches results.

Introduces a new litellmProvider executor wired into the provider registry, supporting OpenAI-SDK chat completions (non-streaming and streaming), JSON-schema response formatting, and tool calling with iteration/timing/cost tracking.

Plumbs LiteLLM through the UI/state layer: provider model syncing (ProviderModelsLoader + query contract), provider metadata/icon, dynamic-provider lists, attachment support, and updates API-key visibility logic so litellm/* models don’t require preconfigured keys.

^{Reviewed by Cursor Bugbot for commit b4268ec. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds LiteLLM as a new AI gateway provider, enabling access to 100+ LLM backends through a LiteLLM proxy server. It follows the existing vLLM pattern throughout: OpenAI SDK with a custom baseURL, dynamic model discovery via /v1/models, env vars LITELLM_BASE_URL/LITELLM_API_KEY, and a full tool-calling loop with streaming support.

• Server-side path is complete: litellmProvider is registered, initialize() discovers models from the proxy, and executeRequest correctly handles streaming, tool calls, forced-tool sequencing, and response-format stripping — all mirroring the vLLM implementation.
• Frontend model discovery pipeline is missing: There is no /api/providers/litellm/models route, no getLitellmProviderModelsContract, no case 'litellm' in requestProviderModels, and ProviderModelsLoader never calls useSyncProvider('litellm') — so discovered models are never synced to the client and won't appear in the model selector.
• updateLiteLLMProviderModels in providers/utils.ts and the 'litellm' branch added to all the right type unions/registries are correct groundwork that will be needed once the API route is added.

Confidence Score: 3/5

Safe to merge for server-side execution, but end-to-end model discovery in the UI is broken without additional files that were not included in this PR.

The server-side execution path (initialize, executeRequest, tool loop, streaming) is complete and correct. However, the client-side model discovery pipeline is entirely absent: no API route, no contract, no query hook case, and no ProviderModelsLoader entry for litellm. Without these, users can type in a model name manually but the model selector will show nothing for LiteLLM, making the feature appear non-functional in the UI.

The four files that need to be added or updated to complete the frontend pipeline: apps/sim/app/api/providers/litellm/models/route.ts (new), apps/sim/lib/api/contracts/providers.ts, apps/sim/hooks/queries/providers.ts, and apps/sim/app/workspace/[workspaceId]/providers/provider-models-loader.tsx.

Important Files Changed

Filename	Overview
apps/sim/providers/litellm/index.ts	New LiteLLM provider implementation mirroring the vLLM pattern; server-side model discovery via /v1/models and full tool-calling loop are correctly implemented, but the frontend model sync pipeline is missing.
apps/sim/app/workspace/[workspaceId]/providers/provider-models-loader.tsx	Not changed by this PR but is missing the litellm entry — no useSyncProvider('litellm') call, causing discovered models to never reach the client-side store or model selector.
apps/sim/providers/models.ts	Adds litellm to PROVIDER_DEFINITIONS, DYNAMIC_MODEL_PROVIDERS, getBaseModelProviders exclusion list, and updateLiteLLMModels helper; defaultModel placeholder 'litellm/generic' is a non-functional model ID.
apps/sim/providers/utils.ts	Adds litellm metadata, updateLiteLLMProviderModels, and getApiKey handler following the vLLM pattern; updateLiteLLMProviderModels is defined but never called from any production path yet.
apps/sim/providers/litellm/utils.ts	Thin wrapper around createOpenAICompatibleStream following the vLLM utils pattern exactly.
apps/sim/providers/registry.ts	Correctly registers litellmProvider in the provider registry alongside other providers.
apps/sim/providers/attachments.ts	Adds litellm to AttachmentProvider union, PROVIDER_SUPPORTED_LABELS, getAttachmentProvider, and isMimeTypeSupportedByProvider; correctly treats it as an image-capable provider.
apps/sim/providers/types.ts	Adds 'litellm' to the ProviderId union type correctly.
apps/sim/stores/providers/store.ts	Adds litellm to the default providers state in the Zustand store.
apps/sim/stores/providers/types.ts	Adds 'litellm' to the ProviderName union type.
apps/sim/blocks/utils.ts	Correctly excludes litellm from the API key requirement check alongside ollama and vllm.
apps/sim/lib/core/config/env.ts	Adds LITELLM_BASE_URL and LITELLM_API_KEY env vars following the same schema pattern as VLLM_BASE_URL/VLLM_API_KEY.
apps/sim/components/icons.tsx	Adds LitellmIcon SVG component with a simple LiteLLM logo representation.

Sequence Diagram

sequenceDiagram
    participant Client as Browser (ProviderModelsLoader)
    participant API as /api/providers/litellm/models ❌ missing
    participant Init as litellmProvider.initialize()
    participant Proxy as LiteLLM Proxy (/v1/models)
    participant Exec as litellmProvider.executeRequest()

    Note over Client,API: Frontend model discovery (NOT implemented)
    Client--xAPI: GET /api/providers/litellm/models (no route exists)

    Note over Init,Proxy: Server-side initialization (works correctly)
    Init->>Proxy: GET /v1/models
    Proxy-->>Init: "{ data: [{ id: 'anthropic/claude-...' }] }"
    Init->>Init: "this.models = ['litellm/anthropic/...']"
    Init->>Init: setProviderModels('litellm', models)

    Note over Client,Exec: Execution flow (works correctly)
    Client->>Exec: "POST /api/providers { provider: 'litellm', model: 'litellm/...' }"
    Exec->>Proxy: "chat.completions.create (OpenAI SDK, baseURL=LITELLM_BASE_URL/v1)"
    Proxy-->>Exec: ChatCompletion / Stream
    Exec-->>Client: ProviderResponse / StreamingExecution

Loading

_{Reviews (1): Last reviewed commit: "fix: add litellm to attachments, provide..." | Re-trigger Greptile}

[greptile-apps]

The defaultModel value 'litellm/generic' is not a real model ID and will produce a confusing error when it is used as the initial default before any models are discovered (e.g. if the proxy is unreachable). Aligning with the vLLM pattern of leaving defaultModel as an empty string avoids sending a bogus model ID to the proxy.

Suggested change

	defaultModel: 'litellm/generic',
	defaultModel: '',

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 475820a. Configure here.}

[RheagalFire]

cc @TheodoreSpeaks @waleedlatif1