Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 100 additions & 4 deletions crates/adapter-zarvis/src/interactive.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1328,6 +1328,30 @@ mod tests {
LineEditor::new(b"> ", 2)
}

#[test]
fn default_monitor_spec_picks_cheap_same_provider_tier() {
// Frontier operator models → a cheaper same-provider model.
assert_eq!(
default_monitor_spec("codex-oauth", "gpt-5.5").as_deref(),
Some("codex-oauth:gpt-5.4-mini")
);
assert_eq!(
default_monitor_spec("openai", "gpt-5.5").as_deref(),
Some("openai:gpt-5-mini")
);
assert_eq!(
default_monitor_spec("anthropic", "claude-opus-4-8").as_deref(),
Some("anthropic:claude-sonnet-4-5")
);
// Already-small operator models → keep them (no downgrade).
assert_eq!(default_monitor_spec("openai", "gpt-5-mini"), None);
assert_eq!(default_monitor_spec("anthropic", "claude-haiku-4-5"), None);
assert_eq!(default_monitor_spec("anthropic", "claude-sonnet-4-5"), None);
// No confident cheap default → keep the operator's model.
assert_eq!(default_monitor_spec("ollama", "llama3"), None);
assert_eq!(default_monitor_spec("gemini", "gemini-pro"), None);
}

#[test]
fn parse_triage_finding_filters_nothing() {
assert!(parse_triage_finding("nothing").is_none());
Expand Down Expand Up @@ -2076,10 +2100,35 @@ pub async fn run(
// previews never accumulate in the operator's context and only escalations
// reach it. Configure a cheaper model via AGENTD_OPERATOR_MONITOR_MODEL;
// otherwise it falls back to the operator's own model.
let monitor_model = std::env::var("AGENTD_OPERATOR_MONITOR_MODEL")
.ok()
.filter(|s| !s.is_empty())
.and_then(|spec| crate::agent::resolve_model_from_spec(&spec).ok());
// Resolve the monitor model (orchestrator-only — that's the only session
// that ambient-ticks). The explicit override wins; otherwise default to a
// cheaper same-provider tier (e.g. mini / sonnet) so the per-tick triage
// doesn't run on the operator's frontier model. A startup health-check
// falls back to the operator's own model when the chosen model can't be
// resolved or doesn't actually answer.
let monitor_model = if is_orchestrator {
let candidate = std::env::var("AGENTD_OPERATOR_MONITOR_MODEL")
.ok()
.filter(|s| !s.is_empty())
.or_else(|| default_monitor_spec(provider_name, &model))
.and_then(|spec| crate::agent::resolve_model_from_spec(&spec).ok());
match candidate {
Some(m) if monitor_model_usable(&m).await => {
tracing::info!(model = %m.model, "operator ambient monitor model");
Some(m)
}
Some(m) => {
tracing::warn!(
model = %m.model,
"ambient monitor model unusable; falling back to operator model"
);
None
}
None => None,
}
} else {
None
};

'outer: loop {
// Wait for a user message — drain order: startup prompt
Expand Down Expand Up @@ -3159,6 +3208,53 @@ If anything qualifies, reply with at most 3 one-line findings, each: '<session i
<what + why, with a short evidence snippet>'. If nothing qualifies, reply with exactly the single \
word: nothing";

/// Default monitor model when `AGENTD_OPERATOR_MONITOR_MODEL` is unset: a
/// cheaper tier on the **same provider** as the operator (so auth/keys are
/// already present), using model names the codebase/provider is known to
/// accept. Returns `None` — keep the operator's own model — when the operator
/// is already on a small model, or the provider has no obvious cheap default
/// (gemini/ollama/unknown), so we never silently pick a non-existent model.
fn default_monitor_spec(provider_name: &str, operator_model: &str) -> Option<String> {
let m = operator_model.to_ascii_lowercase();
if m.contains("mini") || m.contains("nano") || m.contains("haiku") {
return None; // already a small/cheap model
}
match provider_name {
// ChatGPT-account Codex exposes a restricted model set; `gpt-5.4-mini`
// is the small tier (the `*-codex-mini` names are rejected). The
// startup health-check falls back if an account lacks it.
"codex-oauth" => Some("codex-oauth:gpt-5.4-mini".to_string()),
"openai" => Some("openai:gpt-5-mini".to_string()),
"anthropic" if !m.contains("sonnet") => Some("anthropic:claude-sonnet-4-5".to_string()),
// anthropic already on sonnet, or gemini/ollama/unknown → keep operator's.
_ => None,
}
}

/// One-shot liveness check for the chosen monitor model. A wrong model name
/// resolves fine but 400s at call time, which would silently blind the monitor
/// (every triage returns "nothing"). A tiny completion at startup lets us fall
/// back to the operator's own model — which we know works — instead.
async fn monitor_model_usable(m: &crate::agent::ResolvedModel) -> bool {
let messages = vec![Message {
role: Role::User,
content: Content::Text {
text: "Reply with the single word: ok".to_string(),
},
}];
let mut sink = DiscardSink;
crate::provider_watchdog::complete(
m.provider.as_ref(),
&m.model,
"Health check.",
&messages,
&[],
&mut sink,
)
.await
.is_ok()
}

/// Null sink for one-shot completions where we only want the final text.
struct DiscardSink;
impl TextSink for DiscardSink {
Expand Down
4 changes: 2 additions & 2 deletions specs/0022-operator-ambient-loop-runs-as-monitor-subagent.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ Scope: How the Operator's ambient loop scans the fleet and decides what reaches

The ambient loop ([0020](0020-operator-runs-ambiently.md)) no longer injects the fleet snapshot + previews ([0021](0021-operator-ambient-tick-carries-fleet-snapshot.md)) into the Operator's own conversation. Instead each tick runs a **one-shot monitor triage** — a separate, ideally cheaper completion — that judges the (data-only) snapshot + previews off the Operator's context and returns either a concise finding or "nothing". Only a finding becomes an Operator turn; a "nothing" tick never touches the Operator.

The monitor model is configurable via `AGENTD_OPERATOR_MONITOR_MODEL` (falls back to the Operator's own model).
The monitor model is configurable via `AGENTD_OPERATOR_MONITOR_MODEL`. When unset, it **defaults to a cheaper tier on the same provider** as the Operator (so auth is already present): `codex-oauth` → `gpt-5.4-mini`, `openai` → `gpt-5-mini`, `anthropic` → `claude-sonnet-4-5`; already-small Operator models and providers without an obvious cheap tier (gemini/ollama/unknown) keep the Operator's model. A wrong model name resolves fine but 400s at call time (which would silently blind the monitor), so the resolved model is **health-checked once at startup** and falls back to the Operator's own model if it can't be resolved or doesn't answer.

## Reason

Run in the Operator's own session, the now-rich snapshot + previews accumulated in the Operator's persistent conversation: every tick (and every real user turn) ran near the budget ceiling on a frontier model, with stale per-tick snapshots crowding out real conversation and driving compaction. Splitting it makes the bulky, stale, every-5-minutes material live and die in a throwaway triage on a cheap model; the Operator carries *findings*, not *scans*, and only wakes when there's something. The monitor triages mechanically (no user-context), the Operator filters with context.

## Consequences

The Operator's context grows only on real findings (a couple of sentences each, with an evidence snippet + session id) — not on quiet ticks. The triage is a bounded, stateless call (snapshot + previews only), so its cost doesn't grow over time and is cheap when a small model is configured. The Operator's awareness of monitoring becomes structural (system prompt) plus the findings it receives, rather than a pile of no-op receipts. Triage liberality is a prompt dial: too eager pings the Operator for routine activity (cheap, filtered with `noted`); too conservative misses things.
The Operator's context grows only on real findings (a couple of sentences each, with an evidence snippet + session id) — not on quiet ticks. The triage is a bounded, stateless call (snapshot + previews only), so its cost doesn't grow over time and is cheap on the default small model. The Operator's awareness of monitoring becomes structural (system prompt) plus the findings it receives, rather than a pile of no-op receipts. Triage liberality is a prompt dial: too eager pings the Operator for routine activity (cheap, filtered with `noted`); too conservative misses things. The small default model catches high-value "blocked/stuck" findings but is less exhaustive than the frontier model (e.g. it may miss subtler opportunities); override `AGENTD_OPERATOR_MONITOR_MODEL` for more thorough triage.

## Non-Goals

Expand Down
Loading