Models and Budgets

Models, Routing and Budgets

Provider support

Provider implementations include Anthropic, OpenAI, Gemini, Azure OpenAI, Ollama and OpenAI-compatible APIs. Exact available models depend on your credentials, deployments and the selected surface. Vision, image, audio and video tasks need compatible models/providers; a text-only model cannot be routed to a missing capability.

Cascade Auto vs pinned models

With Auto routing, Cascade evaluates task type, capability, benchmark quality, pricing, previous outcomes and availability to choose a suitable model for each tier/subtask. The autoBias setting controls the quality/cost preference: balanced, quality, or cost.

Use the /model picker or specify models.t1, models.t2 and models.t3 in configuration to pin a model. A manually pinned tier model is not swapped merely to stay under a budget.

Benchmark reliability

The benchmark pipeline normalizes different scoring systems rather than treating incompatible percentages and Elo as equivalent. Some committed benchmark source values are explicitly labeled seed-approximation, rather than presenting them as fresh official measurements. Live data may supplement cached data; a bundled snapshot is the fallback.

See benchmark-aggregation.md and the src/core/router implementation.

Cost and token caps

For a capped task, the current run-budget policy normally reserves 20% of the cap for the final answer, leaving around 80% for planning and execution. New work should not consume the final-answer reserve. When necessary, Auto can choose cheaper eligible models and the planner can trim sections/workers while respecting dependencies. Output should say when the budget excluded some planned work.

This is a runtime budget mechanism, not an exact guarantee of the provider's final invoice; model accounting and in-flight usage have uncertainty.

/model          # interactive provider/tier/model picker
/models         # available models
/rate good      # positive outcome feedback
/rate bad       # negative outcome feedback
/why            # rationale behind selection and failover
/cost           # actual/estimated usage and savings
/budget         # view or modify session budget

Use local: true for an OpenAI-compatible/Ollama endpoint only when you genuinely own the inference compute. A remotely rented Ollama machine is not inherently free.

Sources: run-budget.ts, schema.ts.