Auto Model Routing for Autonomous Agents to Reduce Token Usage and Cost

[harshit-lyzr]

Description

Problem

Autonomous agents currently use a single model for their entire execution flow. When a reasoning-capable model is selected, it is applied to every task, including simple operations such as summarization, extraction, formatting, and transformations.

This results in:

• Unnecessary token consumption

• Higher inference costs

• Increased latency for lightweight operations

• Inefficient utilization of model capabilities

Many agent workflows contain a mix of simple and complex tasks, but today all tasks are executed using the same model regardless of complexity.

---

Proposed Solution

Introduce Automatic Model Routing within autonomous agents.

The agent runtime should classify tasks based on complexity and automatically route them to the most appropriate model:

Task Type | Recommended Model -- | -- Summarization | Lightweight model Data extraction | Lightweight model Classification | Lightweight model Text transformation | Lightweight model File search & analysis | Reasoning model Multi-step planning | Reasoning model Decision-making | Reasoning model Tool orchestration | Reasoning model Complex problem solving | Reasoning model

This enables agents to use expensive reasoning models only when necessary while delegating simple tasks to faster and more cost-efficient models.

---

Benefits

• Reduce overall token consumption

• Lower operational costs

• Improve execution speed and latency

• Optimize model utilization

• Maintain reasoning quality for complex workflows

• Improve scalability of autonomous agents

---

Example Workflow

Current Behavior

Selected Model: GPT-5 Reasoning

1. Read document → GPT-5 Reasoning

2. Extract entities → GPT-5 Reasoning

3. Summarize content → GPT-5 Reasoning

4. Make decision → GPT-5 Reasoning

All steps use the same expensive model.

---

Proposed Behavior

Selected Model: GPT-5 Reasoning

1. Read document → Lightweight model

2. Extract entities → Lightweight model

3. Summarize content → Lightweight model

4. Make decision → GPT-5 Reasoning

The reasoning model is reserved only for tasks that require deeper analysis.

---

Implementation Considerations

• Define task complexity classification framework.

• Create a routing layer that determines model selection at runtime.

• Support default task-to-model mappings.

• Allow users to override routing rules through configuration.

• Add observability to track:

  • Model selected per task

  • Token savings

  • Cost savings

  • Latency improvements

• Ensure fallback to the primary model if routing fails.

---

Acceptance Criteria

• Autonomous agents support automatic model routing.

• Lightweight tasks are automatically executed using lower-cost models.

• Reasoning-intensive tasks continue using the configured reasoning model.

• Users can customize routing behavior.

• Token and cost reduction metrics are measurable and exposed in execution logs.

[RiteshTiwari1]

Hi @harshit-lyzr ,

I'd like to work on this. I've hit this exact problem while using GitAgent's SkillFlow system — every step uses the same model regardless of complexity.

Proposed approach:

• Add model? to SkillFlowStep (extends existing channel? pattern in workflows.ts)
• Add model? to SkillMetadata frontmatter (consistent with existing confidence, usage_count fields in skills.ts)
• Add routing to AgentManifest.model in loader.ts (extends existing preferred/fallback pattern)
• Fallback to preferred if no step-level model set
• Leverage existing per-model cost tracker in sdk.ts for observability

Happy to open a draft PR if you can assign this.