Summary
Allow a single Hermes agent profile to route tasks to different models based on task type — e.g., narrative/scripting tasks to Model A, vision/screen-reasoning tasks to Model B — within a single conversational context, with the routing handled transparently by the agent framework.
Problem Statement
Hermes profiles currently assign exactly one model per agent at startup. This works fine for general-purpose agents, but falls short for agents with workflows that span fundamentally different capability domains.
Example use case (Pam Beesly):
- Pam's job is tutorial video creation — she narrates what she sees on screen
- Her best free narration model:
meta-llama/llama-3.3-70b-instruct:free (warm, concise, strong instruction following)
- Her best free screen-reasoning model:
nvidia/nemotron-nano-12b-v2-vl:free (OCR, UI analysis, document understanding — directly built for this)
- These are two different model families solving two different parts of her workflow
- There's no native way to assign both to one profile and have the agent route between them transparently
Today, the only workaround is manual: the user or the agent itself has to manually switch models via a slash command or out-of-band reconfiguration. That's a friction tax on every task that could benefit from mixed-model reasoning.
Proposed Solution
Core Feature: Task-Type Model Routing
Introduce a new config block in config.yaml that maps task categories to models:
model_routing:
default: meta-llama/llama-3.3-70b-instruct:free
vision: nvidia/nemotron-nano-12b-v2-vl:free
coding: qwen/qwen3-next-80b-a3b-instruct:free
# extensible — any task category the agent can self-identify
The routing logic lives in run_agent.py (or a new model_router.py module):
- The agent identifies the task type for each user turn (vision task, narrative task, coding task, etc.)
- The router selects the appropriate model for that task type
- Tool calls, message history, and agent state remain coherent across model switches
- The routing is transparent to the human — no slash commands, no restarts
Key Design Goals
- Backward compatible —
model: in config still works as the single-model alias; model_routing: supersedes it when present
- User-defined task categories — not hardcoded; the profile author defines what categories matter for their agent
- Stateless routing — each model swap is a fresh context window pass; no complex cross-model state to manage
- Tool-call agnostic — the selected model for a task type should be the one with the best tool-matching for that category (e.g., vision model for screenshot analysis)
Routing Trigger Mechanisms
Two options for how the agent decides when to route:
Option A — Agent self-routes (preferred)
The agent is prompted to recognize when a task falls into a specific category and issue an internal routing request. The framework handles the model swap and continues. The agent's system prompt includes the routing map and instructions for when to use each.
Option B — Explicit routing via internal tool
A route_task(task_type: str) internal tool the agent calls to trigger a model switch. More explicit, easier to debug, but requires the agent to be explicitly prompted to use it.
Both are implementable; Option A is cleaner from a UX standpoint, Option B is easier to reason about at a code level.
Alternatives Considered
| Approach |
Why not |
| Subagent delegation (status quo) |
Works but is opaque to the user and requires manual orchestration by the agent or user |
Hot-swap via /model slash command |
Functional but introduces friction on every task boundary — not seamless |
| Per-profile single model with skill-based tool selection |
Doesn't solve the core problem — the model itself may be wrong for certain task types even if tools are available |
Impact
- Non-breaking — existing single-model profiles work exactly as before
- New capability only — agents without
model_routing: configured are unaffected
- Low core footprint — routing logic is a thin layer on top of existing model resolution
- Profile-authors benefit immediately — anyone with a multi-capability agent workflow can use this
Suggested Implementation Owner
This is core framework work — belongs in run_agent.py or a new model_router.py, with routing schema defined in hermes_cli/config.py alongside existing model config. Likely requires coordination between the CLI team and the gateway/platforms team since it affects how agents consume models.
Summary
Allow a single Hermes agent profile to route tasks to different models based on task type — e.g., narrative/scripting tasks to Model A, vision/screen-reasoning tasks to Model B — within a single conversational context, with the routing handled transparently by the agent framework.
Problem Statement
Hermes profiles currently assign exactly one model per agent at startup. This works fine for general-purpose agents, but falls short for agents with workflows that span fundamentally different capability domains.
Example use case (Pam Beesly):
meta-llama/llama-3.3-70b-instruct:free(warm, concise, strong instruction following)nvidia/nemotron-nano-12b-v2-vl:free(OCR, UI analysis, document understanding — directly built for this)Today, the only workaround is manual: the user or the agent itself has to manually switch models via a slash command or out-of-band reconfiguration. That's a friction tax on every task that could benefit from mixed-model reasoning.
Proposed Solution
Core Feature: Task-Type Model Routing
Introduce a new config block in
config.yamlthat maps task categories to models:The routing logic lives in
run_agent.py(or a newmodel_router.pymodule):Key Design Goals
model:in config still works as the single-model alias;model_routing:supersedes it when presentRouting Trigger Mechanisms
Two options for how the agent decides when to route:
Option A — Agent self-routes (preferred)
The agent is prompted to recognize when a task falls into a specific category and issue an internal routing request. The framework handles the model swap and continues. The agent's system prompt includes the routing map and instructions for when to use each.
Option B — Explicit routing via internal tool
A
route_task(task_type: str)internal tool the agent calls to trigger a model switch. More explicit, easier to debug, but requires the agent to be explicitly prompted to use it.Both are implementable; Option A is cleaner from a UX standpoint, Option B is easier to reason about at a code level.
Alternatives Considered
/modelslash commandImpact
model_routing:configured are unaffectedSuggested Implementation Owner
This is core framework work — belongs in
run_agent.pyor a newmodel_router.py, with routing schema defined inhermes_cli/config.pyalongside existing model config. Likely requires coordination between the CLI team and the gateway/platforms team since it affects how agents consume models.