Problem
A routing target is modeled as an LLM endpoint. You can point one at an agent, but
only by wrapping the agent in an OpenAI-compatible adapter yourself — and the
defaults still assume a model:
DEFAULT_DETERMINISTIC_TIER_TIMEOUT_S is 600s, far short of an agent that plans
and executes code
- nothing streams while an agent works, so client idle timeouts fire on a healthy run
- there is no convention for what an agent target returns (usage, model id)
Why it matters
Routing between models and agents is the interesting case: a cheap model for routine
work, an agent when the request needs tools or code execution. We ran a three-target
route (Claude Opus, a literature-search agent, and BioMni) and every agent-specific
concern had to be solved outside Switchyard — an adapter, a bridge, a credential
proxy, and six hand-tuned timeouts.
Proposed behavior
A target kind that says "this is an agent, not a model":
targets:
biomni:
kind: agent
model: gym/biomni-agent
base_url: http://127.0.0.1:9300/v1
With it:
- agent-appropriate timeout defaults, instead of a model's 600s
- no assumption that first-byte latency is small
- a documented response contract so adapters do not each invent one
Notes
Related to #240. That issue lets a classifier pick among arbitrary named targets;
this one asks that those targets can be agents rather than models. They are
complementary — #240 is the routing decision, this is what sits at the end of it.
Problem
A routing target is modeled as an LLM endpoint. You can point one at an agent, but
only by wrapping the agent in an OpenAI-compatible adapter yourself — and the
defaults still assume a model:
DEFAULT_DETERMINISTIC_TIER_TIMEOUT_Sis 600s, far short of an agent that plansand executes code
Why it matters
Routing between models and agents is the interesting case: a cheap model for routine
work, an agent when the request needs tools or code execution. We ran a three-target
route (Claude Opus, a literature-search agent, and BioMni) and every agent-specific
concern had to be solved outside Switchyard — an adapter, a bridge, a credential
proxy, and six hand-tuned timeouts.
Proposed behavior
A target kind that says "this is an agent, not a model":
With it:
Notes
Related to #240. That issue lets a classifier pick among arbitrary named targets;
this one asks that those targets can be agents rather than models. They are
complementary — #240 is the routing decision, this is what sits at the end of it.