Skip to content

operator: triage the ambient fleet scan off-context, escalate only findings#376

Merged
edwin-zvs merged 1 commit into
mainfrom
operator-monitor-subagent
Jun 6, 2026
Merged

operator: triage the ambient fleet scan off-context, escalate only findings#376
edwin-zvs merged 1 commit into
mainfrom
operator-monitor-subagent

Conversation

@edwin-zvs

@edwin-zvs edwin-zvs commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #369/#371. Addresses the context-accumulation problem with the ambient loop.

Problem

The ambient loop runs in the operator's own session, and #371 made each tick rich (fleet snapshot + up to 10 PTY-screen previews). Those got push_msg!'d into the operator's persistent conversation, so:

  • every tick — and every real user turn — ran near the budget ceiling on a frontier model,
  • stale per-tick snapshots (obsolete by the next tick) crowded out the real conversation and drove compaction,
  • zarvis.jsonl grew between compactions.

Per the design discussion: the snapshot/previews are throwaway, and continuity is already handled elsewhere (the delta is adapter state; widgets persist on disk). So the bulk shouldn't live in the operator's context at all.

Change: scan → cheap triage → escalate only findings

Each tick now runs a one-shot monitor triage — a separate completion, ideally on a cheaper model — that judges the data-only snapshot + previews off the operator's context and returns a concise finding or nothing:

  • Quiet ticknothing → the loop continues; the operator is never touched.
  • A finding → a compact, evidenced observation (a sentence per flagged session + a snippet + the session id) becomes the operator turn. The operator never sees the raw previews — it can drill in itself with the id if it wants.

AGENTD_OPERATOR_MONITOR_MODEL selects the monitor model (falls back to the operator's). The triage is stateless and bounded, so its cost doesn't grow over time and is cheap with a small model. The monitor triages mechanically (no user-context); the operator filters with context.

Pieces

  • build_ambient_observation now yields a data-only scan (judgment instructions moved to AMBIENT_MONITOR_SYSTEM); snapshot summary trimmed accordingly.
  • run_ambient_triage (one-shot complete, no tools, discard sink) + parse_triage_finding (reply → compact observation or None).
  • AmbientTick arm: scan → triage → finding | continue 'outer.

Verification

  • parse_triage_finding unit-tested (filters nothing/empty; formats findings); 36 interactive tests pass; full build clean.
  • One live triage against the running fleet returned 3 evidenced findings — including a session blocked on a risky-command approval ("? approve [risky] shell(git add … && git push … && gh pr create …)") and one finished with a release outcome — i.e. the compact findings the operator would see, not the previews. On a quiet fleet it returns nothing and the operator isn't woken.

Spec 0022 added. Code in agentd-adapter-zarvis.

Relationship to 40ec9f6 ("Run Operator ambient loop only while fleet active")

That commit landed on main after this branch was first pushed; this PR is now rebased on top of it. They are complementary, not conflicting (clean rebase, builds, tests pass):

  • 40ec9f6 controls when a tick fires — only while the fleet is active — and drops the interval from 300s to 60s.
  • This PR controls what reaches the operator (only findings) and the per-tick cost (a cheap off-context triage instead of a full operator turn).

Notably, 40ec9f6 alone made the operator run a full frontier turn every 60s while the fleet is active — i.e. the exact context-accumulation / cost problem this PR fixes, made ~5× more frequent by the faster interval. So this PR matters more now. With the 60s cadence, configuring AGENTD_OPERATOR_MONITOR_MODEL to a cheap model is recommended. (Minor: the active-gate and the scan each list() the fleet once per fired tick — a redundant call that could be folded together in a follow-up.)

…ndings

The ambient snapshot + previews had been injected into the operator's own
conversation, so every tick (and every real user turn) ran near the budget
ceiling on a frontier model, with stale per-tick snapshots crowding out real
context and driving compaction.

Now each tick runs a one-shot monitor triage -- a separate, ideally cheaper
completion (AGENTD_OPERATOR_MONITOR_MODEL, falls back to the operator's model)
-- that judges the data-only snapshot + previews OFF the operator's context and
returns a concise finding or "nothing". Only a finding becomes an operator
turn; quiet ticks never touch the operator. So the operator carries findings,
not scans, and only wakes when there's something.

- build_ambient_observation now produces the data-only scan (instructions moved
  to the monitor system prompt); compute_ambient_snapshot summary trimmed.
- run_ambient_triage: one-shot completion, no tools, discard sink; parse_triage
  _finding turns the reply into a compact operator observation or None.
- AmbientTick arm: scan -> triage -> finding | continue (no turn).

Verified live: from a real fleet, the monitor returned 3 evidenced findings --
incl. a session blocked on a risky-command approval and one finished with a
release outcome -- the compact findings the operator would see, not the
previews. Pure parser unit-tested; spec 0022 added.
@edwin-zvs
edwin-zvs force-pushed the operator-monitor-subagent branch from 167afb3 to 5b0ac13 Compare June 6, 2026 17:23
@edwin-zvs
edwin-zvs merged commit 0f681bb into main Jun 6, 2026
1 check passed
@edwin-zvs
edwin-zvs deleted the operator-monitor-subagent branch June 6, 2026 17:27
edwin-zvs added a commit that referenced this pull request Jun 6, 2026
… suppression (#380)

The operator panel showed the operator's response to each ambient/observation
turn ("noted", "Worked", a finding) but not the INPUT that triggered it -- the
observation is a Message event, never rendered to the panel's PTY. So a user
watching the panel saw context-free "noted / Worked / noted ..." and couldn't
tell what was being noted.

Now, before the response streams, the operator echoes the OBSERVATION trigger
into the panel (dim), with the monitor's instruction boilerplate stripped so
only the substance shows -- an ambient monitor finding or a fleet event. The
panel reads question -> answer.

Also removes the dead ambient-turn PTY/message suppression: it keyed on the
stale "ambient operator loop tick" string (the loop's observation became
"ambient fleet monitor flagged" in #376), so it never fired -- and the intent
is now to SHOW responses with their trigger, not hide them. Leaving it was a
trap (fixing its string would have hidden the wanted responses).

The typewriter monolog (#379) is unaffected -- it consumes the response Message
stream; the echo is PTY-only.

- interactive.rs: observation_panel_echo() + emit it before the user Message;
  removed ambient_turn detection + both suppression blocks.
- Test: observation_panel_echo strips the marker/boilerplate, keeps substance,
  returns None for real user input. 38 interactive tests pass. Spec 0024.
edwin-zvs added a commit that referenced this pull request Jun 7, 2026
…widget

Sharpen the operator's surfacing rule. The text monolog is literally the
operator's monolog -- it types over the matrix animation, fades, and the user
has no way to reply. So text is for low-stakes, passing narration only.
Anything the user should act on, decide, or be reliably notified of MUST be an
Operator widget (widgets persist and carry action links).

Also fixes stale/contradictory guidance left by the #376 monitor split:
- The ambient paragraph keyed on "OBSERVATION: ambient operator loop tick",
  which the operator no longer receives (it gets "ambient fleet monitor
  flagged"), so its guidance never fired. Updated the string.
- Removed "Prefer updating/removing compact Operator widgets over chatting,"
  which contradicted the FYI-text guidance; SURFACING is now the single rule.

Guard test asserts the stale string is gone, the new string + principle are
present, and the contradiction stays removed. Spec 0023 updated.
edwin-zvs added a commit that referenced this pull request Jun 7, 2026
…widget (#386)

* operator: text monolog is FYI-only; anything needing a response is a widget

Sharpen the operator's surfacing rule. The text monolog is literally the
operator's monolog -- it types over the matrix animation, fades, and the user
has no way to reply. So text is for low-stakes, passing narration only.
Anything the user should act on, decide, or be reliably notified of MUST be an
Operator widget (widgets persist and carry action links).

Also fixes stale/contradictory guidance left by the #376 monitor split:
- The ambient paragraph keyed on "OBSERVATION: ambient operator loop tick",
  which the operator no longer receives (it gets "ambient fleet monitor
  flagged"), so its guidance never fired. Updated the string.
- Removed "Prefer updating/removing compact Operator widgets over chatting,"
  which contradicted the FYI-text guidance; SURFACING is now the single rule.

Guard test asserts the stale string is gone, the new string + principle are
present, and the contradiction stays removed. Spec 0023 updated.

* ci: re-trigger (previous run hung ~4h)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant