Smart LLM Routing for AI Agents
Most AI agent clients start with one configured model. That is convenient for setup, but it becomes limiting very quickly:
Simple tasks do not need the most expensive model.
Complex coding or reasoning tasks may need a stronger model.
Image, file, browser, or tool-heavy tasks may need a different model path.
Provider outages need fallback.
Customers should not configure every provider, key, model, and routing rule themselves.
Smart routing in 1flowbase solves this by turning routing into a workflow-backed virtual model endpoint.
The external client still calls one model name. Behind that endpoint, 1flowbase can let the main model call a mounted routing tool, switch to a branch LLM, allow that branch LLM to use external tools, and return the result back to the main node as a tool result.
Claude Code / Codex / Cursor / SDK
-> one 1flowbase virtual model endpoint
-> main LLM
-> smart routing tool
-> branch LLM with external tools
-> tool result
-> main LLM final answer
This is different from a hidden LLM router. You can inspect the route, tool call, branch model execution, token usage, latency, failure status, and final result in the run log.
The previous multimodal pattern mounts a vision model as a tool. The branch LLM reads uploaded image blocks and returns visual context to the main model.
Smart routing is more powerful:
- The branch LLM is allowed to take over the routed task.
- The branch LLM can call enabled external tools.
- The branch result returns to the main node as a structured tool result.
- The client still sees one normal model endpoint.
In other words:
Multimodal mounting: the branch model reads media.
Smart routing: the branch model can work with tools and return the finished sub-result.
Use smart routing when the routed task needs tool access, not only another model's opinion.
Good use cases:
- Route screenshot or file inspection to a model that can call
Read,Glob, browser, OCR, or internal tools. - Route simple classification, formatting, or extraction to a cheap or local model.
- Route complex coding, planning, or review to a stronger model.
- Route private data to a local or domestic provider.
- Route provider failures to a fallback model.
- Build one product-facing model endpoint while keeping provider complexity inside 1flowbase.
Avoid smart routing when:
- A normal single-model answer is enough.
- You do not want branch models to use tools.
- You cannot tolerate extra latency.
- You have not reviewed the branch model's tool permissions.
In this demo, the external client is Claude Code. The user only sends a normal request:
uploads\test-01.png
看看这张图代码在哪里？
Claude Code does not need to know which model should inspect the image or which tools should be used. It only calls the 1flowbase virtual model endpoint.
Inside 1flowbase, the main LLM sees that image/file inspection should be handled by a mounted smart routing tool. That tool switches to the configured branch LLM, lets it use external tools, and returns the result to the main LLM.
A minimal workflow looks like this:
Start
-> Main LLM: user-facing reasoning and final answer
-> Mounted tool: image_llm
-> Branch LLM: task-specific model
-> External tools: Read / Glob / browser / app tools
-> Tool result: structured sub-result
-> Main LLM: final response
The main model does not have to do every step itself. It can delegate a subtask to the smart routing tool, then use the returned result to answer the user.
Open the main LLM node settings, enable Mount LLM, then add a tool registration.
For the demo:
Tool name: image_llm
Tool identifier: tool_1
Use a description that tells the main model when this route should be used. For example:
Use this tool when the user asks to inspect an image, file path, screenshot, code location, UI screenshot, or uploaded media. The routed model may use enabled external tools to read files, inspect context, and return a structured result.
The important part is not the exact name. The important part is that the description gives the main model a clear routing policy.
In the tool registration:
- Enable Allow branch LLM.
- Set Tool mode to
Smart routing. - Enable open external tools.
These switches change the behavior:
- Allow branch LLM lets the mounted tool call downstream LLM nodes.
- Smart routing tells 1flowbase that this is a routed work handoff, not a simple text transform.
- open external tools lets the branch LLM use the external tools you expose to it.
Only enable external tools for routes that actually need them. A branch model with tool access is more capable, but it also needs clearer boundaries.
Choose a branch LLM that matches the task.
Examples:
image_llm -> vision or UI inspection model
code_llm -> stronger coding model
cheap_llm -> cheap/local model for simple tasks
private_llm -> local or domestic provider for private data
fallback_llm -> fallback provider when the primary model fails
For the screenshot demo, the branch model can inspect the uploaded path, call tools such as Read or Glob, and return the relevant file locations and implementation details.
The branch result should be concrete:
- files inspected
- likely code location
- relevant component or route
- evidence found
- next action or question
That makes the main LLM's final answer grounded in what the branch actually did.
Configure Claude Code, Codex, Cursor, OpenCode, Cline, Continue, LibreChat, or another client to call the published 1flowbase endpoint.
Conceptually:
Base URL: your 1flowbase endpoint base URL
API key: your 1flowbase app key
Model: the published virtual model name
Protocol: Claude-compatible Messages API or OpenAI-compatible API
Then send a normal user request.
The client does not configure DeepSeek, GLM, Claude, Gemini, OpenRouter, fallback, branch tools, or model-specific routing rules. It just calls one model endpoint.
Open Log, select the run, then open Conversation log and the Track tab.
In the demo run, the log shows:
- normal tool attempts such as
GlobandRead image_llm Smart routing Interceptedimage_llm Smart routing Executed successfully- the tool input with
type: "visible_internal_llm_tool" - media reference such as
uploads/test-01.png - the final answer in the run details panel
This is the core value of 1flowbase. The router is not a black box. You can see which route was used, what the routed model received, which tools were available, and what came back to the main node.
For production use, make the routing decision explicit. A branch or router node can output something like:
{
"task_type": "image_code_location",
"risk_level": "medium",
"reasoning_depth": "medium",
"context_need": "workspace_files",
"tool_access_required": true,
"selected_route": "image_llm",
"confidence": 0.82,
"reason": "The user provided an uploaded image path and asked where the corresponding code is located."
}This gives you something to audit later:
- Did the router choose the right branch?
- Did it route because the task was hard, visual, private, or tool-heavy?
- Did the branch model use the right tools?
- Did the route save cost or only add latency?
Smart routing should not mean "always use the cheapest model first."
In agent workflows, a cheaper model may need more retries, more tool loops, and more human correction. That can make the final outcome slower or more expensive.
Use the run log to compare:
model selected
route reason
input tokens
output tokens
latency
tool calls
fallback events
final answer quality
The right metric is not only token price. The right metric is outcome quality, cost, and reliability for the full task.
The main model never calls the smart routing tool.
Strengthen the tool description. Mention the exact trigger: image path, screenshot, file inspection, code location, private data, fallback, or complex task.
The branch model cannot use tools.
Check that open external tools is enabled and that the external tools are actually exposed to the routed branch.
The wrong branch model is selected.
Make the routing instruction more specific. Separate task type, risk, context need, privacy need, and tool access need instead of relying only on token length.
The route works but the answer is vague.
Require a structured branch result: files inspected, evidence, reasoning, and final sub-result.
The workflow is too slow.
Use smart routing only for tasks that need it. Keep ordinary chat, simple formatting, and low-risk summaries on the main model path.
The workflow is hard to trust.
Add route decision output and replay a small set of real tasks after changing the route prompt or branch model.
This guide is relevant if you are searching for:
smart LLM routing
LLM router for AI agents
dynamic model routing
model router
AI agent model routing
per-message model selection
per-task model selection
OpenAI-compatible endpoint
OpenAI-compatible proxy
Claude Code model routing
Codex model routing
Cursor model routing
workflow-backed model endpoint
multi-model agent workflow
model routing observability
cost attribution for LLM routing
route decision trace
one endpoint many models
Your customer configures one model endpoint.
1flowbase runs the model routing workflow behind it.
The routed model can use tools, return a tool result, and stay visible in the trace.
If you are building AI agents, coding agents, or model-powered products, smart routing lets you keep the client simple while moving provider choice, tool access, fallback, and cost visibility into a workflow you control.