Skip to content

run_in_terminal: promote sync command to background after idle silence#316166

Merged
meganrogge merged 5 commits into
mainfrom
merogge/idle-silence-test
May 13, 2026
Merged

run_in_terminal: promote sync command to background after idle silence#316166
meganrogge merged 5 commits into
mainfrom
merogge/idle-silence-test

Conversation

@meganrogge
Copy link
Copy Markdown
Collaborator

@meganrogge meganrogge commented May 13, 2026

Re-land of #315885 on a standalone branch for isolated eval testing.

What this does

If a synchronous run_in_terminal call produces no terminal output for N ms (default 60s), a new idleSilence race candidate wins the foreground race and promotes the execution to background. The process is never killed — the model gets the terminal ID + output so far and can get_terminal_output, send_to_terminal, or kill_terminal.

Changes (3 files)

  • runInTerminalTool.ts: New idleSilence race candidate using RunOnceScheduler + onData listener. Refactors _buildInputNeededSteeringText from mentionTimeout: boolean to hungHint: 'none' | 'timeout' | 'idleSilence' discriminator with per-mode wording.
  • terminalChatAgentToolsConfiguration.ts: New setting chat.tools.terminal.idleSilenceTimeoutMs (default 60000, 0 disables, experimental).
  • runInTerminalTool.test.ts: 3 unit tests for steering text across all hint modes.

Why this is safe

  • Commands that produce output regularly (npm install, cargo build, etc.) reset the timer and never trip.
  • The async path is unchanged — it already has OutputMonitor idle detection.
  • Listener + scheduler owned by raceCleanup DisposableStore — disposed when any other race candidate wins.

Megan Rogge added 2 commits May 11, 2026 18:05
If a synchronous run_in_terminal call produces no output for N ms, win the
foreground race with a new idleSilence candidate that mirrors the existing
timeout handler: promote the execution to background, return the terminal
ID + output collected so far, append a steering hint. The process is never
killed.

Gated on chat.tools.terminal.idleSilenceTimeoutMs (default 60000, 0
disables). Listener and scheduler are owned by the existing raceCleanup
DisposableStore so they go away when another candidate wins. Async
(waitStrategy === 'idle') path is unchanged.

Fixes #315884
Replace the boolean mentionTimeout parameter on _buildInputNeededSteeringText with a 'none' | 'timeout' | 'idleSilence' discriminator so the idle-silence promotion result no longer reuses the timeout wording. Add focused unit tests covering each mode.
Copilot AI review requested due to automatic review settings May 13, 2026 01:31
@meganrogge meganrogge self-assigned this May 13, 2026
@meganrogge meganrogge marked this pull request as draft May 13, 2026 01:31
@meganrogge meganrogge added this to the 1.121.0 milestone May 13, 2026
@meganrogge
Copy link
Copy Markdown
Collaborator Author

/requires-eval-assessment terminalbench2 gpt-5.4,claude-opus-4.6,claude-opus-4.7

@meganrogge meganrogge added the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
@meganrogge meganrogge changed the title Merogge/idle silence test run_in_terminal: promote sync command to background after idle silence May 13, 2026
@vs-code-engineering
Copy link
Copy Markdown
Contributor

⏳ Queued vscode build for 6127d25b49b46fcdbda2e7e99e7c764d80e197cb (step 1/2).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the terminal chat agent “run in terminal” tool to support an idle-silence path: when a foreground/sync command produces no output for a configurable duration, the tool returns early, moves the execution to a background terminal, and provides updated steering guidance to the model.

Changes:

  • Add a new configuration setting chat.tools.terminal.idleSilenceTimeoutMs to control idle-silence promotion timing (0 disables).
  • Implement idle-silence promotion logic in RunInTerminalTool and adjust input-needed steering text to distinguish 'none' | 'timeout' | 'idleSilence'.
  • Add unit tests validating the steering text content across the new “hung hint” modes.
Show a summary per file
File Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/electron-browser/runInTerminalTool.test.ts Adds tests for steering text behavior across none/timeout/idle-silence modes.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/common/terminalChatAgentToolsConfiguration.ts Introduces the new idleSilenceTimeoutMs setting with schema/description.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/runInTerminalTool.ts Implements idle-silence promotion and updates steering text API/call sites.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 2

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 13, 2026

Base: d4a82f0b Current: 1a2e0e0e

No screenshot changes.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@meganrogge meganrogge added ~requires-eval-assessment Evals will be run and will generate a report upon completion and removed ~requires-eval-assessment Evals will be run and will generate a report upon completion labels May 13, 2026
@vs-code-engineering
Copy link
Copy Markdown
Contributor

⏳ Queued vscode build for ac12483253ad0e87ab389a990e66836878ade865 (step 1/2).

@meganrogge meganrogge removed the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
Resolve conflict in runInTerminalTool.ts: keep idleSilence race type + try/finally cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@meganrogge meganrogge added the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
@vs-code-engineering
Copy link
Copy Markdown
Contributor

⏳ Queued vscode build for f47921fc80f79d8d21a2443985e1125b8be2d736 (step 1/2).

@vs-code-engineering
Copy link
Copy Markdown
Contributor

🚀 Queued eval-assessment publish build for d1e7b002a190e508170eda45bd36ef37329002bc (step 2/2).

@vs-code-engineering
Copy link
Copy Markdown
Contributor

🔬 Queued eval-assessment benchmark for 9232a54894.

Results will be posted back here when the run completes.

@vs-code-engineering
Copy link
Copy Markdown
Contributor

✅ Eval-assessment build published.

@vs-code-engineering vs-code-engineering Bot removed the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
The revert in main (#316165) removed the idle-silence feature. This merge
keeps all idle-silence changes from the branch: race candidate, config
setting, steering text refactor, and tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@microsoft microsoft deleted a comment from vs-code-engineering Bot May 13, 2026
@microsoft microsoft deleted a comment from vs-code-engineering Bot May 13, 2026
@microsoft microsoft deleted a comment from vs-code-engineering Bot May 13, 2026
@meganrogge meganrogge added the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
@vs-code-engineering
Copy link
Copy Markdown
Contributor

⏳ Queued vscode build for 4b8e564e65d062f978c09202e52d1f9903ac9fbe (step 1/2).

@vs-code-engineering
Copy link
Copy Markdown
Contributor

🚀 Queued eval-assessment publish build for 1a2e0e0ecf41f752a5bb205396c0866e2f882840 (step 2/2).

@vs-code-engineering
Copy link
Copy Markdown
Contributor

🔬 Queued eval-assessment benchmark for 70a42806f0.

Results will be posted back here when the run completes.

@vs-code-engineering
Copy link
Copy Markdown
Contributor

✅ Eval-assessment build published.

@vs-code-engineering vs-code-engineering Bot removed the ~requires-eval-assessment Evals will be run and will generate a report upon completion label May 13, 2026
@meganrogge meganrogge marked this pull request as ready for review May 13, 2026 16:05
@meganrogge meganrogge enabled auto-merge (squash) May 13, 2026 16:05
@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

🧪 Results

@meganrogge meganrogge merged commit 3a58860 into main May 13, 2026
25 checks passed
@meganrogge meganrogge deleted the merogge/idle-silence-test branch May 13, 2026 18:11
@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

🧪 Results

@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

🧪 Results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants