Fix repetition detection signature to survive ref churn between snapshots

[lmorchard]

Current state

Pilo detects repeated-action loops via checkAndHandleRepeatedAction (packages/core/src/webAgent.ts:1119-1200). The signature it tracks is built at webAgent.ts:1484-1486:

const signature = `${actionOutput.action}:${actionOutput.ref || ""}:${
  actionOutput.value || ""
}`;

So fill:E12:hello and fill:E12:world are different signatures. Two thresholds:

• Warning threshold (maxRepeatedActions + 1, default 3): on the third identical signature, append a user-message warning and force a fresh snapshot.
• Abort threshold (maxRepeatedActions + 2, default 4): emit TASK_ABORTED with reason "Excessive repetition…".

The gap

The ref (e.g., E42) is part of the signature, but refs are regenerated on every new page snapshot (webAgent.ts:503-505, approvedRefs.clear(); the ariaTree renumbers from E1 each snapshot). A logical "click Submit button" might be ref E42, then E58, then E13 across iterations.

Practical consequences:

• A genuine action-level loop ("agent repeatedly tries to submit a form that silently doesn't work") goes undetected because each attempt has a different ref.
• The detector only catches "stuck on literally the same ref ID" — a much narrower failure mode than the actual one.
• The hard-abort at threshold 4 is then triggered mostly by edge cases where refs don't change between iterations (typically pages where no DOM mutation happened — i.e., where the action did nothing — exactly the case the detector should catch, but only by accident).

Also worth flagging:

• maxRepeatedActions is a confusing config name. With maxRepeatedActions: 2, the warning fires at count 3 and abort at count 4. The constant value is two-less than what it appears to mean.
• The warning threshold and "force fresh snapshot" are the same threshold, which is backwards: if the agent is in a loop, the next snapshot is likely the same page anyway. The fresh snapshot doesn't help the agent escape the loop.
• There's no detection of stagnant pages (same URL/title/major content for N steps regardless of action) — a useful signal separate from repeated actions.

Proposed scope

1. Replace ref-based signature with a canonical-action signature

Drop the ref from the signature. Normalize the value. Possible signature shape:

function actionSignature(actionOutput: ActionResult): string {
  const action = actionOutput.action;
  const value = (actionOutput.value ?? "").toString().toLowerCase().trim();
  // For tools where the target identity matters more than the ref, fold in
  // a normalized form (e.g., accessible name of the element) from the snapshot
  // if available. For now: action + value is dramatically better than action+ref+value.
  return `${action}:${value}`;
}

If we want richer matching (so "click element whose role+name is Submit" hashes the same whether it's ref E42 or E58), the tool result needs to surface the target's identifying info (role + accessible name) alongside the ref. That requires a small change to performActionWithValidation to thread element identity into ActionResult. Worth doing as a follow-up; the action+value change alone closes the largest gap.

2. Rename the config option

maxRepeatedActions → repetitionWarningCount (or similar). The value should mean what it says: the number of identical actions before the agent gets a warning. Add the abort threshold as a separate option (e.g., repetitionAbortCount, default = warningCount + 1).

Provide backward compat for one release: read both names, log a deprecation if the old name is used, then drop.

3. Stagnant-page tracking (separate signal)

Track "the page's URL + title + a hash of the structural top-level elements" per step. If unchanged for N steps (default 3), inject a nudge:

"The page hasn't changed in the last 3 steps. Consider whether your actions are taking effect, or whether a different approach would make progress."

This is separate from repeated actions and catches a different failure mode (the agent keeps trying things, none of them change the page).

4. Decouple warning from forced snapshot

When the warning fires, do not force a fresh snapshot in the same iteration — the snapshot is already up-to-date from the previous turn. The warning is enough.

Implementation notes

• Be careful: a legitimate workflow is "scroll down a few pages on an infinite-scroll feed" — same action, varying ref. The action+value change preserves this case (each scroll has different pages? actually same pages arg, so they'd dedupe). Decide whether scroll should be exempted from the detector or whether the signature should include something position-dependent for scroll.
• Tests: extend the error handling describe block in webAgent.test.ts:1119+ with cases for the new signature behavior. Add explicit cases for: same logical click across multiple snapshots (was previously undetected; now should be), legitimate scroll repetition (should not abort).
• The warning user-message should not stack: if the warning has been fired in the last 2 iterations, don't fire again. Currently the warning re-fires every iteration past threshold.

Acceptance criteria

• Same logical action repeated 3+ times in a row across multiple snapshots is detected (previously was not).
• Renamed config option works; deprecated old option logs a warning.
• Stagnant-page detector fires after N stagnant steps, injects a single nudge per stagnation streak.
• Warning no longer forces a fresh snapshot in the same iteration.
• Tests cover all four behavioral changes.

Effort estimate

1-2 days. The signature change is small; the threading of element identity into ActionResult is the largest piece. Stagnant-page detection can ship in a follow-up if time-constrained.

Related issues

Independent.

Files likely affected

• packages/core/src/webAgent.ts (checkAndHandleRepeatedAction, signature builder, ExecutionState)
• packages/core/src/tools/webActionTools.ts (optional: surface element identity in ActionResult)
• packages/core/src/config/defaults.ts (renamed config)
• packages/core/test/webAgent.test.ts (error handling and new repetition detection blocks)