packages ai models

Models

Active contributors: Mario Zechner, kt, Armin Ronacher

Purpose

The model registry is the catalog of every model pi-ai can talk to, generated into packages/ai/src/models.generated.ts and loaded into memory by packages/ai/src/models.ts. It gives consumers a typed Model<TApi> for any known provider and model id, plus cost calculation and thinking-level helpers. The catalog is never hand-maintained: it is regenerated by packages/ai/scripts/generate-models.ts, and per repo rule (AGENTS.md, also in Patterns and conventions) you must never edit packages/ai/src/models.generated.ts directly.

The Model type

Every entry in the catalog is a Model (defined in packages/ai/src/types.ts):

Field	Meaning
id / name	Model identifier and display name
api	Which provider implementation serves it, e.g. anthropic-messages, openai-completions
provider	The registry provider key, e.g. anthropic, deepseek, openrouter
baseUrl	Upstream endpoint; may contain {VAR} placeholders resolved from env for Cloudflare
reasoning	Whether the model supports thinking
thinkingLevelMap	Maps pi thinking levels (off .. max) to provider-specific values; null marks a level unsupported
input	Supported input modalities, text and optionally image
cost	USD per million tokens: input, output, cacheRead, cacheWrite
contextWindow / maxTokens	Token limits used for overflow detection and generation caps
featured	Flagship model surfaced above non-featured models in pickers
headers	Static headers to send with requests
compat	Provider-specific compatibility overrides (OpenAI Completions, Responses, Anthropic Messages)

Directory layout

packages/ai/
├── src/
│   ├── models.ts             # runtime registry, cost calc, thinking-level helpers
│   ├── models.generated.ts   # generated catalog: 1172 models, 31 providers
│   └── cache-pricing.ts      # Anthropic cache cost multipliers
└── scripts/
    └── generate-models.ts    # regenerates models.generated.ts from upstream catalogs

How the registry works

packages/ai/src/models.ts builds a Map<provider, Map<modelId, Model>> from the exported MODELS object at module load. The public surface:

• getModel(provider, modelId) returns a Model typed to the exact api of that entry.
• getProviders() and getModels(provider) list the registry contents.
• calculateCost(model, usage, overrides?) turns per-million-token prices into a Usage["cost"] breakdown from the actual token counts.
• getSupportedThinkingLevels(model) and clampThinkingLevel(model, level) derive which of the seven levels a model supports from reasoning and thinkingLevelMap, and clamp a requested level to the nearest supported one.
• supportsFastMode(model) flags the GPT-5.4/5.5/5.6 Codex models used for fast mode.
• modelsAreEqual(a, b) compares by id and provider.

The registry only contains built-in models. Custom providers and models are added at runtime by the coding agent from ~/.prime/agent/models.json (documented in packages/coding-agent/docs/models.md) through registerApiProvider and the model list in packages/coding-agent/src/core/model-registry.ts.

Generation pipeline

packages/ai/scripts/generate-models.ts runs on every build (npm run build calls npm run generate-models first). It:

1. Fetches https://models.dev/api.json, the primary source for Anthropic, Google, OpenAI, Groq, Cerebras, and Bedrock models.
2. Fetches the OpenRouter public catalog for xAI and other providers that models.dev does not cover.
3. Fetches the Vercel AI Gateway catalog for OpenAI-compatible models.
4. Merges all sources with models.dev taking priority (first model per provider+id wins, so later sources only fill gaps), filters to tool-call-capable models, and drops unsupported variants (e.g. Google live, deep-research, and computer-use models).
5. Applies per-model overrides: thinking-level maps, compat objects, cache pricing corrections, and context-window fixes verified against live APIs.
6. Writes packages/ai/src/models.generated.ts with providers and models sorted for deterministic output, then prints per-provider model counts.

The current catalog has 1172 models across 31 providers: amazon-bedrock, anthropic, azure-openai-responses, cerebras, cloudflare-ai-gateway, cloudflare-workers-ai, deepseek, fireworks, github-copilot, google, google-vertex, groq, huggingface, kimi-coding, minimax, minimax-cn, mistral, moonshotai, moonshotai-cn, openai, openai-codex, opencode, opencode-go, openrouter, prime-inference, vercel-ai-gateway, xai, xiaomi, xiaomi-token-plan-ams, xiaomi-token-plan-cn, xiaomi-token-plan-sgp, zai.

Cache pricing

packages/ai/src/cache-pricing.ts centralizes Anthropic-style prompt cache costs. getAnthropicCacheCosts(inputCost, duration) returns read cost at 0.1x input and write cost at 1.25x input for 5-minute retention or 2x for 1-hour retention. getAnthropicCacheWriteCost can blend costs from actual cache_creation usage when the API reports separate ephemeral 5m and 1h token counts. hasStandardAnthropicCachePricing checks whether a model's stored cacheWrite price matches the expected multiplier, which lets providers detect models whose upstream pricing is already cache-aware. The Anthropic and Bedrock providers use these helpers to fill in cacheRead and cacheWrite costs and to report accurate usage.

How model discovery reaches the user

The coding agent layers user-facing discovery on top of this registry:

• packages/coding-agent/src/core/model-registry.ts imports getModels / getProviders, overlays custom models, and resolves which providers are authenticated via getEnvApiKey and OAuth.
• packages/coding-agent/src/cli/list-models.ts and the /model slash command present the merged list; featured models float to the top of pickers.
• OAuth providers can rewrite models after login via modifyModels (e.g. GitHub Copilot sets a session base URL in packages/ai/src/utils/oauth/github-copilot.ts).
• prime-inference models are fetched separately by the coding agent (packages/coding-agent/src/core/prime-inference-models.ts) and merged in at runtime.

Entry points for modification

• Change model metadata or add models: edit the fetch and override logic in packages/ai/scripts/generate-models.ts, then run npm run generate-models from packages/ai/. Never edit packages/ai/src/models.generated.ts.
• Change cost accounting: packages/ai/src/models.ts (calculateCost) and packages/ai/src/cache-pricing.ts.
• Change thinking-level semantics: getSupportedThinkingLevels and clampThinkingLevel in packages/ai/src/models.ts, plus the ThinkingLevel types in packages/ai/src/types.ts.

Key source files

File	Role
packages/ai/src/models.ts	Runtime registry, cost calculation, thinking-level helpers
packages/ai/src/models.generated.ts	Generated catalog (1172 models, 31 providers)
packages/ai/scripts/generate-models.ts	Generation pipeline from models.dev, OpenRouter, and AI Gateway
packages/ai/src/cache-pricing.ts	Anthropic cache cost multipliers
packages/ai/src/types.ts	Model, ThinkingLevel, ThinkingLevelMap, CostOverrides

Related pages

• AI package overview: how models connect to providers and streams
• Providers: the implementations that consume Model entries
• Provider login and model selection: user-facing login and model pickers
• Coding agent overview: custom models via ~/.prime/agent/models.json
• Glossary: term definitions