Skip to content

outputMonitor: fix two false-positive families pausing the agent loop#315485

Merged
meganrogge merged 1 commit into
microsoft:mainfrom
ba-work:ba-work/output-monitor-input-pattern-315476
May 11, 2026
Merged

outputMonitor: fix two false-positive families pausing the agent loop#315485
meganrogge merged 1 commit into
microsoft:mainfrom
ba-work:ba-work/output-monitor-input-pattern-315476

Conversation

@ba-work
Copy link
Copy Markdown
Contributor

@ba-work ba-work commented May 9, 2026

Fixes #315476.

This change addresses two related false-positive families in RunInTerminalTool's input-needed heuristic that pause the agent loop on benign terminal output. Both surfaced live: the $Ri family on log lines ending in : or ?, and the Vyo family on every command return when the shell uses an oh-my-zsh-style git-aware PS1 prompt.

Family 1 — detectsInputRequiredPattern's broad fallbacks match log output

detectsInputRequiredPattern augmented detectsHighConfidenceInputPattern's strict pattern set with two over-broad fallbacks:

/: +$/                              // any line ending colon + spaces
/\? *(?:\([a-z\s]+\))? +$/i     // any line ending question + spaces

These fire on benign log output like:

[INFO] Starting:
Last Command:
find: /tmp/x: No such file:
Are you sure you want to do this?

The strict guard at the original call site is shell-integration's isActive flag, which is only true when the foreground process is actively holding stdin open. _handleIdleState was using the broad pattern unconditionally, with no isActive evidence.

Fix: split the function into two:

  • detectsInputRequiredPattern(line): strict-only — delegates to detectsHighConfidenceInputPattern. Safe to call from any path, including unconditionally on the last line of a finished command.
  • detectsLikelyInputRequiredPattern(line): strict + broad fallbacks. Should only be called when an out-of-band signal already confirms the process is awaiting input (isActive === true).

Update the callers:

  • _handleIdleState: switch to the strict detectsInputRequiredPattern. Eliminates the log-line false positives.
  • _waitForIdle: keep using the broad detectsLikelyInputRequiredPattern. This call is gated by recentlyIdle && isActive === true, which provides the out-of-band evidence the broad fallbacks need.

Family 2 — detectsHighConfidenceInputPattern's paren-default rule matches every oh-my-zsh prompt

detectsHighConfidenceInputPattern contained this rule:

/:\s*\([^)]*\) +$/      // "package name: (test) "

intended for npm-init / yarn-init style prompts. Because \s* allows zero spaces between : and (, it also matches every git-aware shell prompt:

➜  myrepo git:(main)
➜  vscode git:(ba-work/output-monitor)

(oh-my-zsh's robbyrussell theme; bash with __git_ps1; etc.) Live trigger rate: every shell-prompt return, on every benign command, on every machine running oh-my-zsh. The agent loop pauses for ~2 minutes (the _handleTimeoutState recovery window) before resuming on every command, making multi-step tasks unusable.

Fix: tighten \s*\s+ so at least one space is required between : and (. npm-init prompts always render a space (name: (default)); shell-prompt PS1 themes never do (git:(main)). Single-byte regex change, no behavioural change to the npm-init / yarn-init matches.

Tests

  • Update Line ends with colon to use the strict path and assert that bare Enter your name: , File to overwrite: , and log lines like Last Command: , [INFO] Starting: , find: /tmp/x: No such file: , error: subprocess returned: , and 2025-05-09 12:34:56 INFO Starting: all return false under the strict detectsInputRequiredPattern.
  • Update detects trailing questions similarly.
  • Add a new suite for detectsLikelyInputRequiredPattern covering strict-set parity, bare-colon/question matching, non-prompt rejection, and the documented false positives (log lines with colons) so the broad surface stays understood.
  • Add regression cases in detectsHighConfidenceInputPattern asserting that oh-my-zsh / git-aware prompts return false while npm-init prompts continue to return true.
  • Add the same oh-my-zsh regression cases in detectsInputRequiredPattern (strict) and detectsLikelyInputRequiredPattern so the rule's flow through both call paths is pinned.
  • Add three end-to-end OutputMonitor integration tests asserting onDidDetectInputNeeded does NOT fire for (a) log lines ending in : , (b) lines ending in ? , (c) oh-my-zsh git-aware shell prompts. These pin the wiring at _handleIdleState, not just the regex behaviour in isolation.

How to verify

Family 2 (oh-my-zsh prompts): in the integrated terminal with shell integration enabled, on a machine using oh-my-zsh's robbyrussell theme (or any git-aware PS1 that renders git:(branch) with no space after the colon), run any command from agent mode that returns to the prompt:

cd ~/anywhere && git status

Without the fix: the agent pauses for ~2 minutes after every command return. With the fix: the agent continues immediately.

Family 1 (broad-fallback log lines): from agent mode, run a slow-ish command whose visible buffer ends in : while the command is still being polled (the strict path is what fires onDidDetectInputNeeded; running command output, not the post-prompt return, is what _handleIdleState evaluates):

printf 'Compiling: \n' && sleep 2

Without the fix: the agent pauses with the input-needed wrapper because detectsInputRequiredPattern's broad /: +$/ fallback matches the trailing : . With the fix: _handleIdleState calls the strict path, which no longer carries those fallbacks, and the wrapper does not fire.

Scope

This PR is intentionally limited to the two false-positive families called out in #315476. While reading the surrounding code I noticed detectsGenericPressAnyKeyPattern is invoked with outputTail rather than outputLastLine (line ~370), which is inconsistent with sibling line-oriented detectors. That predates this PR and is out of scope here; happy to file a separate fix if a maintainer agrees it's worth addressing.

Copilot AI review requested due to automatic review settings May 9, 2026 18:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refines the terminal OutputMonitor input-needed heuristics used by the chat agent terminal tooling to reduce false positives that pause the agent loop on benign output (e.g., log lines ending in : / ? and git-aware shell prompts like git:(main)).

Changes:

  • Split “input-needed” detection into a strict path (detectsInputRequiredPattern → high-confidence only) and a broader, gated path (detectsLikelyInputRequiredPattern).
  • Tighten the parenthesized-default regex to avoid matching oh-my-zsh / git-aware prompts (:\s*:\s+).
  • Expand unit/integration-style tests to cover the reported false-positive families and regressions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/monitoring/outputMonitor.ts Splits strict vs broad prompt detection, updates idle polling gating, and tightens the parenthesized-default regex.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/outputMonitor.test.ts Adds regressions and new suites validating strict vs likely behavior and prompt false-positive avoidance.

@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from c198177 to 1367c62 Compare May 9, 2026 18:40
@ba-work ba-work requested a review from Copilot May 9, 2026 18:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from 1367c62 to e712877 Compare May 9, 2026 18:57
@ba-work ba-work requested a review from Copilot May 9, 2026 19:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from e712877 to 0aae89a Compare May 9, 2026 19:15
@ba-work ba-work requested a review from Copilot May 9, 2026 19:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Fixes microsoft#315476.

This change addresses two related false-positive families in
RunInTerminalTool's input-needed heuristic that pause the agent loop
on benign terminal output.

### Family 1 — ``'s broad fallback regexes match log output

`detectsInputRequiredPattern` ($Ri minified) augmented `Vyo`'s
strict pattern set with two over-broad fallbacks:

  /: +$/                                  (any line ending colon + spaces)
  /\? *(?:\([a-z\s]+\))? +$/i         (any line ending question + spaces)

These fire on log output like:

  [INFO] Starting:
  Last Command:
  find: /tmp/x: No such file:
  Are you sure you want to do this?  (rhetorical)

The strict guard at the call site is shell-integration's `isActive`
flag, which is only true when the foreground process is actively
holding stdin open. `_handleIdleState` (line ~388/408) was using
the broad pattern unconditionally, with no isActive evidence.

Fix: split the function into two:

  - detectsInputRequiredPattern(line): strict-only — delegates to
    detectsHighConfidenceInputPattern. Safe to call from any path,
    including unconditionally on the last line of a finished command.
  - detectsLikelyInputRequiredPattern(line): strict + broad fallbacks.
    Should only be called when an out-of-band signal already confirms
    the process is awaiting input (isActive === true).

Update the callers:

  - _handleIdleState (lines ~388, ~408): switch to the strict
    detectsInputRequiredPattern. Eliminates the log-line false
    positives that were pausing the agent.
  - _waitForIdle (line ~532): keep using the broad
    detectsLikelyInputRequiredPattern. This call is gated by
    `recentlyIdle && isActive === true`, which provides the
    out-of-band evidence the broad fallbacks need.

### Family 2 — `Vyo`'s paren-default rule matches every oh-my-zsh prompt

`Vyo` (detectsHighConfidenceInputPattern) contained a rule:

  /:\s*\([^)]*\) +$/      // "package name: (test) "

intended for npm-init / yarn-init style prompts. Because `\s*`
allows zero spaces between `:` and `(`, this also matches every
git-aware shell prompt:

  ➜  myrepo git:(main)
  ➜  vscode git:(ba-work/output-monitor)

(oh-my-zsh's robbyrussell theme; bash with __git_ps1; etc.) Live
trigger rate: every shell-prompt return, on every benign command,
on every machine running oh-my-zsh. The agent loop pauses for
~2 minutes (the _handleTimeoutState recovery window) before
resuming on every command, making multi-step tasks unusable.

Fix: tighten `\s*` -> `\s+` so at least one space is required
between `:` and `(`. npm-init prompts always render a space
("name: (default)"); shell-prompt PS1 themes never do
("git:(main)"). Single-byte regex change, no behavioural change to
the npm-init / yarn-init matches.

### Tests

- Update test 'Line ends with colon (strict path)' to assert that
  bare "Enter your name: ", "File to overwrite: ", and log lines
  like "Last Command: ", "[INFO] Starting: ", "find: /tmp/x: No
  such file: ", "error: subprocess returned: ", and
  "2025-05-09 12:34:56 INFO Starting: " all return FALSE under the
  strict detectsInputRequiredPattern.
- Update test 'detects trailing questions (strict path does not match
  bare ? prompts)' similarly.
- Add new suite for detectsLikelyInputRequiredPattern with parity
  tests, bare-colon/question matching, non-prompt rejection, and
  documented false positives (e.g. log lines with colons).
- Add regression cases in detectsHighConfidenceInputPattern asserting
  that oh-my-zsh / git-aware prompts return FALSE while npm-init
  prompts continue to return TRUE.
- Add regression cases in detectsInputRequiredPattern (strict) and
  detectsLikelyInputRequiredPattern asserting the same for the
  oh-my-zsh case (since the strict rule flows through both paths).

### How to verify

Reproduce in the integrated terminal with shell integration enabled
on a machine using oh-my-zsh's robbyrussell theme:

  cd ~/anywhere && git status

Without the fix: agent pauses for ~2 minutes after the command
returns. With the fix: agent continues immediately.
@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from 0aae89a to 14ba5fb Compare May 9, 2026 19:39
@ba-work ba-work requested a review from Copilot May 9, 2026 19:41
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@meganrogge meganrogge assigned meganrogge and unassigned hediet May 11, 2026
@meganrogge meganrogge added this to the 1.121.0 milestone May 11, 2026
Copy link
Copy Markdown
Collaborator

@meganrogge meganrogge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@meganrogge
Copy link
Copy Markdown
Collaborator

Good catch on detectsGenericPressAnyKeyPattern being invoked with outputTail rather than outputLastLine — that does look inconsistent with the sibling detectors. Please go ahead and file a separate issue/PR for it. Thanks for flagging it and for keeping this PR focused.

@meganrogge meganrogge enabled auto-merge (squash) May 11, 2026 15:30
@meganrogge meganrogge merged commit 44551a9 into microsoft:main May 11, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RunInTerminalTool: OutputMonitor.detectsInputRequiredPattern fallback regexes false-positive on benign command output, pausing the agent loop

5 participants