Critic and Replan

Critic and Replan

A second model watches the first. Every N tool calls during a goal run, the GoalRunner fires a sibling call to a "critic" that judges progress and steers the loop when needed. This is what catches stuck loops, premature completions, and drifts away from the goal.

Source: src/main/goal-runner.ts:runCritic().

---

How it fires

After each batch of tool calls, the runner increments stepsSinceLastCritic. When the counter reaches criticIntervalSteps (default 5, configurable per goal), the runner pauses the main loop and calls the critic.

The critic is a non-streaming call to a (usually different) provider with a strict prompt:

System: You are a critic evaluating an autonomous AI agent's progress toward a goal.
        Reply with EXACTLY ONE of these tokens on the first line:
          PROGRESSING — clear forward movement
          STUCK — repeating or thrashing
          ACHIEVED — goal appears complete, ready for verification
          MISLED — drifted away from the goal
        On the second line, give a one-sentence reason. Nothing else.

User:   GOAL: <goal>
        SUCCESS CRITERION: <criterion-humanized>
        RECENT STEPS:
        [12] write_to_terminal(text=ssh user@host…) → ok: <preview>
        [13] read_terminal_output(pane_id=…) → ok: <preview>
        [14] write_to_terminal(text=systemctl status…) → ok: <preview>
        [15] read_terminal_output(pane_id=…) → ok: <preview>
        Verdict:

The critic sees the last criticInterval × 2 steps (so 10 steps for a default interval of 5) to spot patterns.

---

The four verdicts

Verdict	Runner response
PROGRESSING	No-op. Loop continues silently.
STUCK	Inject a system message: "CRITIC: You appear to be stuck. Reason: X. Step back and try a different approach. Consider: are you using the right tool? Is the assumption you've been operating under wrong? Could a simpler or different angle work? If you've genuinely exhausted ideas, call abort_with_report."
ACHIEVED	Short-circuit verification. Sets pendingClaim with rationale "Critic believes goal achieved: X." Next loop iteration runs the [[Success-Criteria
MISLED	Inject a re-anchor: "CRITIC: You've drifted from the goal. Reason: X. ORIGINAL GOAL: . Refocus on the goal. Ignore tangents."

Unrecognized verdicts default to PROGRESSING (don't interrupt a working agent on a parse error).

---

Why this matters

Without the critic, an autonomous agent that gets stuck can burn its entire wall-clock budget repeating a doomed approach — same broken cert edit four times, or polling for output that's never coming. The critic detects "stuck" within a single firing window (5 steps default) and either redirects or aborts.

STUCK is the biggest win. Before this phase, the only outs were the wall-clock cap (slow) or the model spontaneously realizing it was stuck (unreliable). With the critic, "stuck" is detected within ~5 steps and the loop is redirected — or aborted if the model agrees it has nothing left to try.

ACHIEVED is the second. Sometimes the model has clearly done the work but hasn't noticed (or is being overly cautious). The critic notices, short-circuits to verification, and the goal can complete one turn earlier.

---

Configuring the critic

When Starting-a-Goal, two fields control critic behavior:

criticIntervalSteps

Default 5. Set to 3 for tight oversight, 10 for less interruption, 0 to disable entirely.

Transient tools (claim_complete, abort_with_report) are excluded from the step count — an attempt to finish doesn't trip the critic prematurely.

criticProviderId

Optional. By default the critic uses the same provider as the main loop. Set to a cheaper / faster provider (Claude Haiku, GPT-4o-mini, a local Mistral) to make critics cheap.

A typical setup:

• Main provider: Claude Sonnet (smart, expensive)
• Critic provider: Claude Haiku (cheap, fast, fine for yes/no/stuck/achieved)

Configure both as separate providers in AI-Providers, then pass the critic provider's ID when starting goals.

---

Critic events in the dashboard

Critic firings emit a critic event over the goal:event IPC channel. The Goal-Dashboard renders these in the bottom critic rail:

critic:progressing  agent is methodically working through deploy steps
critic:stuck        agent has retried the same nginx config edit 4 times with same error
critic:achieved     server now responds 200, goal should be ready for verification
critic:misled       agent is exploring kubernetes when goal was about local nginx

Color-coded by verdict (green/yellow/emerald/orange). Reads like a story — useful for debugging "what was the AI thinking when it claimed complete?"

---

Limits & gotchas

• Critic accuracy scales with model quality. A tiny critic model may misfire (STUCK on healthy progress, MISSED the actual stuck-ness). Tune criticIntervalSteps higher for low-quality critics.
• Critic interruptions are visible to the main model. Inject messages appear in the conversation, so the next assistant turn knows the critic intervened. Usually helpful (the model adjusts); occasionally jarring (the model justifies its prior approach to the critic and continues).
• Critic doesn't see tool results in detail, only step summaries. If a result was paginated or truncated, the critic only sees the preview. For deeper verification, rely on Success-Criteria and Vision-Verification instead.
• STUCK doesn't auto-abort. The system message asks the model to try a different angle or call abort_with_report. The model decides. If it persists, the wall-clock cap is the backstop.

---

See also

• Goal-Runner-Overview — the loop the critic interrupts
• Success-Criteria — what ACHIEVED short-circuits to
• Goal-Dashboard — where critic events surface in the UI
• AI-Providers — set up a cheap second provider for cost-efficient critics
• Goal-Runner-Internals — the runCritic() implementation