feat(ai/models): enrich response with context_window + cost

[sweetmantech]

Summary

api's GET /api/ai/models now returns two extra fields on each model entry:

• context_window (integer) — max tokens per request
• cost ({input, output}) — per-million-token pricing in USD

Both are sourced from the public models.dev catalog and merged into the gateway response. Documented in recoupable/docs#188 (merged).

Why

Pre-step for cutting open-agents' model picker over to api. The frontend reads both fields:

• apps/web/lib/models.ts filters out models with no context_window
• apps/web/lib/model-options.ts surfaces cost in the model selector

Without these, redirecting useModels() to api strips the model picker.

Architecture

TDD'd in three small focused helpers:

File	Responsibility
lib/ai/parseModelsDevMetadata.ts	pure unknown → Map<id, {context_window, cost}>
lib/ai/fetchModelsDevMetadata.ts	750ms-bounded fetch + parse
lib/ai/enrichGatewayModel.ts	pure, non-mutating merge

lib/ai/getAvailableModels.ts orchestrates: gateway models + metadata fetched in parallel, then each non-embed model is mapped through enrichment.

Best-effort enrichment

Every failure mode of fetchModelsDevMetadata (timeout, non-2xx, malformed JSON, network throw) returns an empty map — the api response stays available with un-enriched models. Today's clients aren't affected.

TDD

• parseModelsDevMetadata.test.ts — 8 cases (well-formed entry, namespace handling, missing-id fallback, partial cost, non-positive context, malformed providers/models)
• fetchModelsDevMetadata.test.ts — 4 cases (200, non-200, throw, JSON parse error)
• enrichGatewayModel.test.ts — 4 cases (no metadata, context-only, cost-only, no mutation)
• getAvailableModels.test.ts updated — 5 cases now mock both gateway + metadata

28/28 ai tests pass. Format + lint:check clean.

Test plan

• CI green
• Preview deployment: GET /api/ai/models returns models with context_window + cost populated for entries that exist in models.dev
• Field-by-field comparison vs sandbox.recoupable.com/api/models response

🤖 Generated with Claude Code

---

Summary by cubic

Adds context and pricing to model listings so the frontend can filter by max tokens and show costs. GET /api/ai/models now returns enriched models and still works if models.dev is down.

• New Features

  • Adds context_window (max tokens) and cost {input, output} (USD per million) per model, sourced from models.dev.
  • Applies enrichment to non-embed models when metadata is available.

• Refactors

  • Fetches @ai-sdk/gateway models and models.dev metadata in parallel; on failure, returns plain gateway models. Fixes typing in enrichGatewayModel to accept gateway entries.
  • Introduces helpers to parse, fetch (750ms timeout), and non-mutating merge of metadata; extracts isRecord into its own utility.

^{Written for commit 51ee19e. Summary will update on new commits.}

Summary by CodeRabbit

• New Features
  • Models now display additional information including context window and cost details, automatically sourced from an external service. This enrichment provides users with better visibility into model capabilities, constraints, and pricing information, helping them make more informed decisions when selecting models.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
api	Ready	Preview	May 5, 2026 1:59pm

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces utilities to enrich gateway models with metadata fetched from the models.dev API. It adds parsing, type-guarding, and enrichment functions that conditionally augment models with context window and cost information while gracefully handling fetch failures.

Changes

Model Metadata Enrichment

Layer / File(s)	Summary
Data Shape lib/ai/parseModelsDevMetadata.ts, lib/ai/enrichGatewayModel.ts	ModelsDevMetadata interface defines optional context_window and cost fields. GatewayModelLike constrains enrichable inputs to objects with an id property.
Parsing & Type Guards lib/ai/isRecord.ts, lib/ai/parseModelsDevMetadata.ts	isRecord type guard safely narrows unknown payloads to plain objects. parseModelsDevMetadata defensively extracts metadata from a nested provider/model structure, validating numeric fields and constructing a Map keyed by provider/model ids.
Fetching & Enrichment lib/ai/fetchModelsDevMetadata.ts, lib/ai/enrichGatewayModel.ts	fetchModelsDevMetadata fetches from models.dev/api.json with a 750ms timeout and returns a Map (empty on any failure). enrichGatewayModel clones input models and conditionally copies context_window and cost when present in metadata.
Integration lib/ai/getAvailableModels.ts	getAvailableModels now concurrently fetches gateway models and metadata, filters non-embed models, and enriches each via enrichGatewayModel.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

✨ Models grow richer with data from the cloud,
Context and cost now wear metadata proud,
Silent graceful failures, no thunder or sound,
Enriched gateway gateways on solid ground! 🌟

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Solid & Clean Code	⚠️ Warning	Cost validation missing non-negative checks. The cost.input/output lack >= 0 validation that context_window has, risking invalid negative pricing in API output.	Add >= 0 validation to cost.input and cost.output in parseModelsDevMetadata.ts (lines 43-44)

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/ai-models-enrich-context-cost

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

3 issues found across 8 files

Confidence score: 4/5

• This PR looks safe to merge with minimal risk: the reported items are moderate-to-low severity (5/10, 4/10, 3/10) and mostly input-validation/style hardening rather than clear breakage.
• The most significant issue is in lib/ai/parseModelsDevMetadata.ts: context_window should be validated as a positive integer, otherwise malformed metadata could be enriched and lead to incorrect model limits downstream.
• Also in lib/ai/parseModelsDevMetadata.ts, negative cost.input/cost.output values should be rejected to prevent invalid pricing metadata from being accepted; the test-file size note in lib/ai/__tests__/parseModelsDevMetadata.test.ts is maintainability-focused and non-blocking.
• Pay close attention to lib/ai/parseModelsDevMetadata.ts, lib/ai/__tests__/parseModelsDevMetadata.test.ts - tighten numeric validation for metadata fields and address test file-size policy compliance.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="lib/ai/parseModelsDevMetadata.ts">

<violation number="1" location="lib/ai/parseModelsDevMetadata.ts:37">
P2: Validate `context_window` as a positive integer before enrichment.</violation>

<violation number="2" location="lib/ai/parseModelsDevMetadata.ts:45">
P2: Reject negative `cost.input`/`cost.output` values when parsing metadata.</violation>
</file>

<file name="lib/ai/__tests__/parseModelsDevMetadata.test.ts">

<violation number="1" location="lib/ai/__tests__/parseModelsDevMetadata.test.ts:1">
P3: Custom agent: **Enforce Clear Code Style and Maintainability Practices**

This new test file exceeds the repository’s 100-line file-size limit.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Client as Frontend Client
    participant API as API Route (/api/ai/models)
    participant GAM as getAvailableModels()
    participant Gateway as @ai-sdk/gateway
    participant FMDM as fetchModelsDevMetadata()
    participant ModelsDev as models.dev
    participant PMDM as parseModelsDevMetadata()
    participant EGM as enrichGatewayModel()

    Note over Client,EGM: GET /api/ai/models — Model Listing Request

    Client->>API: GET /api/ai/models

    Note over API,GAM: NEW: Parallel fetch pattern

    API->>GAM: getAvailableModels()

    GAM->>Gateway: gateway.getAvailableModels()
    GAM->>FMDM: fetchModelsDevMetadata()

    activate Gateway
    activate FMDM

    Note over FMDM: NEW: 750ms timeout with AbortController

    FMDM->>ModelsDev: fetch(https://models.dev/api.json)
    ModelsDev-->>FMDM: API.json payload

    alt Response ok & valid JSON
        FMDM->>PMDM: parseModelsDevMetadata(data)
        PMDM-->>FMDM: Map<id, {context_window, cost}>
        FMDM-->>GAM: Metadata map
    else Non-200, timeout, or parse error
        FMDM-->>GAM: Empty Map (best-effort)
    end

    Gateway-->>GAM: List of gateway models

    deactivate FMDM
    deactivate Gateway

    Note over GAM: NEW: Enrich non-embed models

    GAM->>GAM: Filter embed models
    GAM->>EGM: enrichGatewayModel(model, metadataMap)

    Note over EGM: NEW: Non-mutating merge

    alt Metadata found for model
        EGM-->>GAM: Model with context_window + cost
    else No metadata
        EGM-->>GAM: Unchanged model
    end

    GAM-->>API: Enriched model list
    API-->>Client: 200 JSON with context_window + cost

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Reject negative cost.input/cost.output values when parsing metadata.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/ai/parseModelsDevMetadata.ts, line 45:

<comment>Reject negative `cost.input`/`cost.output` values when parsing metadata.</comment>

<file context>
@@ -0,0 +1,58 @@
+
+      if (
+        isRecord(modelValue.cost) &&
+        typeof modelValue.cost.input === "number" &&
+        typeof modelValue.cost.output === "number"
+      ) {
</file context>

[cubic-dev-ai]

P2: Validate context_window as a positive integer before enrichment.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/ai/parseModelsDevMetadata.ts, line 37:

<comment>Validate `context_window` as a positive integer before enrichment.</comment>

<file context>
@@ -0,0 +1,58 @@
+
+      if (
+        isRecord(modelValue.limit) &&
+        typeof modelValue.limit.context === "number" &&
+        modelValue.limit.context > 0
+      ) {
</file context>

[cubic-dev-ai]

P3: Custom agent: Enforce Clear Code Style and Maintainability Practices

This new test file exceeds the repository’s 100-line file-size limit.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At lib/ai/__tests__/parseModelsDevMetadata.test.ts, line 1:

<comment>This new test file exceeds the repository’s 100-line file-size limit.</comment>

<file context>
@@ -0,0 +1,106 @@
+import { describe, it, expect } from "vitest";
+import { parseModelsDevMetadata } from "@/lib/ai/parseModelsDevMetadata";
+
</file context>

[sweetmantech]

SRP - new lib for isRecord

[sweetmantech]

3-way comparison: PR preview ↔ open-agents production ↔ docs

Smoke-tested against https://api-git-feat-ai-models-enrich-context-cost-recoup.vercel.app/api/ai/models (commit 51ee19e0). Compared against open-agents production https://sandbox.recoupable.com/api/models and the merged docs spec at https://developers.recoupable.com/api-reference/ai/models.

Coverage on the PR preview

• 192 models total
• 189 (98%) include context_window
• 184 (95%) include cost
• 200 unauth — endpoint stays public per spec ✓
• The 3–8 un-enriched models (e.g. minimax/minimax-m2.5-highspeed, openai/gpt-image-1-mini, openai/gpt-image-2) are simply absent from the upstream models.dev catalog. Graceful degradation working as designed — they return without context_window / cost, matching the spec's additionalProperties: true schema.

Single-model side-by-side (alibaba/qwen-3-14b)

This PR's preview:

{
  "id": "alibaba/qwen-3-14b",
  "name": "Qwen3-14B",
  "description": "Qwen3 is the latest generation of large language models in Qwen series…",
  "pricing": { "input": "0.00000012", "output": "0.00000024" },
  "specification": { "specificationVersion": "v2", "provider": "deepinfra", "modelId": "alibaba/qwen-3-14b" },
  "modelType": "language",
  "context_window": 40960,
  "cost": { "input": 0.06, "output": 0.24 }
}

Open-agents production (same model):

{
  "id": "alibaba/qwen-3-14b",
  "name": "Qwen3-14B",
  "description": "Qwen3 is the latest generation of large language models in Qwen series…",
  "pricing": { "input": "0.00000012", "output": "0.00000024" },
  "specification": { "specificationVersion": "v3", "provider": "deepinfra", "modelId": "alibaba/qwen-3-14b" },
  "modelType": "language",
  "context_window": 40960,
  "cost": { "input": 0.06, "output": 0.24 }
}

vs docs spec (recoupable/docs#188 merged)

Field	Docs says	api preview	Match
id, name, description, modelType	string	string	✓
pricing, specification	object (additionalProperties: true)	objects	✓
context_window	integer	40960	✓
cost	{input: number, output: number}	{input: 0.06, output: 0.24}	✓
200 response	{models: [...]}	{models: [...]}	✓
500 response	{message: string}	(handler returns this on error; not exercised live)	✓
Auth	"No authentication required"	confirmed 200 unauth	✓

Result: zero divergence from the docs spec.

vs open-agents production

Field	open-agents	api preview	Diff
id, name, description, pricing, modelType	identical	identical	none
context_window, cost (when present)	identical numeric values	identical	none
specification.specificationVersion	"v3"	"v2"	api uses an older @ai-sdk/gateway version — outside this PR's scope. Field-level fix is just pnpm up @ai-sdk/gateway in api.
Model count	193	192	catalog drift (different @ai-sdk/gateway versions surface different upstream sets): api has openai/gpt-5.4-pro, openai/gpt-image-* that open-agents lacks; open-agents has perplexity/sonar*, meituan/longcat-flash-*, zai/glm-4.6v-flash that api lacks.

What this PR fixes vs what remains

• Closed by this PR: the two fields the open-agents model picker reads (context_window filters models below user's token budget; cost populates the price column). Cutover blocker for the model picker is gone.
• Outside this PR: @ai-sdk/gateway version pin drift causes specificationVersion and the model-catalog delta. A separate small PR can bump api's gateway dep to match open-agents.

Test sessions cleanup

This endpoint is read-only; no rows created during the smoke test.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

lib/ai/parseModelsDevMetadata.ts (1)

17-56: 🏗️ Heavy lift

Split parser internals into small helpers to keep this function within guideline limits.

parseModelsDevMetadata is doing traversal + id normalization + context parsing + cost parsing in one block. Please extract at least metadata extraction and model-id normalization helpers to reduce size and nesting.

♻️ Proposed refactor sketch

 export function parseModelsDevMetadata(data: unknown): Map<string, ModelsDevMetadata> {
   const map = new Map<string, ModelsDevMetadata>();
   if (!isRecord(data)) return map;

   for (const [providerKey, providerValue] of Object.entries(data)) {
     if (!isRecord(providerValue)) continue;
     if (!isRecord(providerValue.models)) continue;

     for (const [modelKey, modelValue] of Object.entries(providerValue.models)) {
       if (!isRecord(modelValue)) continue;
-      const rawId = typeof modelValue.id === "string" ? modelValue.id : modelKey;
-      const modelId = rawId.includes("/") ? rawId : `${providerKey}/${rawId}`;
-      const meta: ModelsDevMetadata = {};
-      // inline parsing...
+      const modelId = toModelId(providerKey, modelKey, modelValue);
+      const meta = extractMetadata(modelValue);
       if (meta.context_window !== undefined || meta.cost !== undefined) {
         map.set(modelId, meta);
       }
     }
   }

   return map;
 }

As per coding guidelines, **/*.{js,ts,tsx,jsx,py,java,cs,go,rb,php}: “Flag functions longer than 20 lines” and “Keep functions small and focused.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/ai/parseModelsDevMetadata.ts` around lines 17 - 56, Split
parseModelsDevMetadata by extracting two helpers: implement a
normalizeModelId(providerKey: string, modelKey: string, modelValue: unknown):
string that returns the correct rawId and prefixes providerKey when needed (used
where modelId is computed), and implement extractModelMeta(modelValue: unknown):
ModelsDevMetadata | undefined that encapsulates the context_window and cost
parsing (move the isRecord checks and numeric validations into this helper and
return undefined when no metadata). Then simplify parseModelsDevMetadata to only
traverse provider/model entries, call normalizeModelId to get modelId and call
extractModelMeta to get meta, and only map.set(modelId, meta) when meta is
returned; keep existing symbol names (parseModelsDevMetadata, modelId, meta) to
make the change local and reduce nesting.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/ai/parseModelsDevMetadata.ts`:
- Around line 41-47: The cost assignment currently accepts any numeric values;
update the guard in parseModelsDevMetadata (the block that checks
isRecord(modelValue.cost) and typeof modelValue.cost.input/output) to also
verify both modelValue.cost.input and modelValue.cost.output are non-negative
(>= 0) before assigning meta.cost; if either is negative, do not set meta.cost
(or explicitly skip/omit it) so invalid negative USD pricing is not propagated.

---

Nitpick comments:
In `@lib/ai/parseModelsDevMetadata.ts`:
- Around line 17-56: Split parseModelsDevMetadata by extracting two helpers:
implement a normalizeModelId(providerKey: string, modelKey: string, modelValue:
unknown): string that returns the correct rawId and prefixes providerKey when
needed (used where modelId is computed), and implement
extractModelMeta(modelValue: unknown): ModelsDevMetadata | undefined that
encapsulates the context_window and cost parsing (move the isRecord checks and
numeric validations into this helper and return undefined when no metadata).
Then simplify parseModelsDevMetadata to only traverse provider/model entries,
call normalizeModelId to get modelId and call extractModelMeta to get meta, and
only map.set(modelId, meta) when meta is returned; keep existing symbol names
(parseModelsDevMetadata, modelId, meta) to make the change local and reduce
nesting.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dd433c78-d661-495a-b395-19754b84b65f

📥 Commits

Reviewing files that changed from the base of the PR and between d525c6d and 51ee19e.

⛔ Files ignored due to path filters (5)

• lib/ai/__tests__/enrichGatewayModel.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
• lib/ai/__tests__/fetchModelsDevMetadata.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
• lib/ai/__tests__/getAvailableModels.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
• lib/ai/__tests__/isRecord.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
• lib/ai/__tests__/parseModelsDevMetadata.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**

📒 Files selected for processing (5)

• lib/ai/enrichGatewayModel.ts
• lib/ai/fetchModelsDevMetadata.ts
• lib/ai/getAvailableModels.ts
• lib/ai/isRecord.ts
• lib/ai/parseModelsDevMetadata.ts

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Validate cost values as non-negative before adding metadata.

cost.input/cost.output are accepted as any numbers. Negative values would propagate invalid USD pricing into API output.

🛡️ Proposed fix

       if (
         isRecord(modelValue.cost) &&
         typeof modelValue.cost.input === "number" &&
-        typeof modelValue.cost.output === "number"
+        typeof modelValue.cost.output === "number" &&
+        modelValue.cost.input >= 0 &&
+        modelValue.cost.output >= 0
       ) {
         meta.cost = { input: modelValue.cost.input, output: modelValue.cost.output };
       }

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	if (
	isRecord(modelValue.cost) &&
	typeof modelValue.cost.input === "number" &&
	typeof modelValue.cost.output === "number"
	) {
	meta.cost = { input: modelValue.cost.input, output: modelValue.cost.output };
	}
	if (
	isRecord(modelValue.cost) &&
	typeof modelValue.cost.input === "number" &&
	typeof modelValue.cost.output === "number" &&
	modelValue.cost.input >= 0 &&
	modelValue.cost.output >= 0
	) {
	meta.cost = { input: modelValue.cost.input, output: modelValue.cost.output };
	}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/ai/parseModelsDevMetadata.ts` around lines 41 - 47, The cost assignment
currently accepts any numeric values; update the guard in parseModelsDevMetadata
(the block that checks isRecord(modelValue.cost) and typeof
modelValue.cost.input/output) to also verify both modelValue.cost.input and
modelValue.cost.output are non-negative (>= 0) before assigning meta.cost; if
either is negative, do not set meta.cost (or explicitly skip/omit it) so invalid
negative USD pricing is not propagated.

[cubic-dev-ai]

0 issues found across 4 files (changes from recent commits).

_{Requires human review: Auto-approval blocked by 3 unresolved issues from previous reviews.}

[sweetmantech]

Skipping the cubic + coderabbit suggestions to reject negative cost values and enforce Number.isInteger on context_window. Data source is the public models.dev catalog — a negative price or fractional context window would be a data-quality bug upstream, and silently filtering those entries hides it rather than surfacing it. The existing isRecord + typeof === "number" guards already cover the realistic failure mode (missing/malformed entries). Adding the extra checks is defensive code for scenarios that don't happen.