Terminal agent: improve send_to_terminal output capture and add waitForOutput

[meganrogge]

Summary

Improves send_to_terminal to capture more useful output and adds a waitForOutput mode for interactive programs.

Changes

Output capture improvements

• Marker-scoped output: Registers an xterm marker before sending text so output is scoped to just the response, not stale output from previous commands
• Increased default output from 5 → 20 lines
• Increased default wait from 100ms → 2000ms to give programs time to respond
• Cancellation-aware timeouts: All timeout() calls now pass the cancellation token so tool cancellation returns promptly

New waitForOutput parameter

• When true, waits for the terminal to become idle (no new output for 2s) before returning
• Critical for interactive programs (games, REPLs, text adventures) where the response arrives asynchronously
• 30s max wait with cancellation support
• Added to schema with description guiding the LLM on when to use it

Model description update

• Updated modelDescription to tell the LLM about the 20-line default and waitForOutput option

Testing

• Added schema validation test for waitForOutput
• Added waitForOutput=true idle-wait test using runWithFakedTimers
• Updated mock execution to support registerMarker and onData
• Preserved all existing multiline/heredoc tests from main

Eval impact

Addresses ~3 terminalbench failures where the agent interacted with interactive programs but got stale/insufficient output (blind-maze-explorer, play-zork, decode-braille).

Fixes #312383

[Copilot]

Pull request overview

Improves the send_to_terminal chat agent tool to better support interactive terminal programs by optionally waiting for terminal output to become idle before returning output.

Changes:

• Added waitForOutput?: boolean input parameter to the tool schema and invocation params.
• Implemented idle-wait logic based on terminal onData activity and scoping output via a start marker.
• Updated/added tests for the new schema parameter and waitForOutput behavior.

Show a summary per file

File	Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/sendToTerminalTool.ts	Adds waitForOutput support, marker-scoped output, and idle-wait polling based on terminal data events.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/sendToTerminalTool.test.ts	Extends mocks for onData/marker support and adds tests for schema + wait-for-idle path.

Copilot's findings

Comments suppressed due to low confidence (2)

src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/sendToTerminalTool.ts:397

• _waitForIdleOutput creates a CancellationTokenSource from token but never cancels it, and the polling timeout(pollIntervalMs) isn't token-aware. This can be simplified by using token.isCancellationRequested directly and/or awaiting timeout(pollIntervalMs, token) so cancellation is observed immediately (rather than up to the next poll interval).

		const cts = new CancellationTokenSource(token);
		const dataListener = execution.instance.onData(() => {
			lastDataTime = Date.now();
		});

		try {
			while (!cts.token.isCancellationRequested && waited < maxWaitMs) {
				await timeout(pollIntervalMs);
				waited += pollIntervalMs;

src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/sendToTerminalTool.ts:410

• When waitForOutput is true, _waitForIdleOutput returns getOutput(execution.instance, startMarker) without a lastNLines limit; if startMarker is undefined this can return up to 60k characters from the entire buffer. To avoid oversized tool responses (and token/latency issues), consider applying a line limit (eg last 20/50 lines) or making the limit explicit/configurable for the waitForOutput path as well.

		return getOutput(execution.instance, startMarker ?? undefined);
	}

• Files reviewed: 2/2 changed files
• Comments generated: 2

[Copilot]

invoke now receives a cancellation token but the default (waitForOutput=false) path uses await timeout(2000) without passing the token, so tool cancellation won't interrupt the 2s delay. Consider using the token-aware timeout(millis, token) (and handling CancellationError if needed) so cancelling a chat/tool invocation returns promptly.

This issue also appears in the following locations of the same file:

• line 389
• line 409

[anthonykim1]

@meganrogge This make sense. So previously, was the start marker attached to previous response/ earlier in terminal buffer? Or maybe even no marker and we were just reading the last X number of terminal buffer.