Evaluate Agent Host custom terminal tool vs SDK shell tool for chat terminal UX parity

[anthonykim1]

For Agent Host Copilot sessions, would we be "Good" relying completely on CLI SDK's built-in shell tool, or continue to invest in custom terminal tool.

Numbers for interaction with Copilot terminal: https://github.com/microsoft/vscode-internalbacklog/issues/8351

The reason this matters: VS Code's terminal is not just stdout/stderr. It is where users look when a command is slow, hung, waiting for input, asking for a secret, starting a server, or doing something surprising. If Agent Host only relies on the SDK shell tool, commands may still run, but we may lose "tight integration" and bunch of UX flows that chat had with integrated terminal.

Current switch:

• chat.agentHost.customTerminalTool.enabled=false: Copilot SDK sessions rely on the SDK shell behavior.
• chat.agentHost.customTerminalTool.enabled=true: Agent Host overrides the SDK built-in shell tools with VS Code/Agent Host terminal-backed tools.

Important runtime context: Copilot CLI/runtime shell execution has moved away from node-pty-backed shell sessions toward direct child_process.spawn with stdio pipes. That is a good direction for predictable command output, but it also means the chat terminal experience can deviate quite a bit from our local agent harness atm.

• Gaps are noted/listed down below in pros/cons section.

Also awkward: AHP itself treats terminals as first-class subscribable resources: https://microsoft.github.io/agent-host-protocol/guide/terminals.html.

What we need to decide:

• Is the SDK/runtime shell tool enough for VS Code Agent Host sessions?
• Do we need the Agent Host custom terminal tool for VS Code-quality terminal UX?
• Or do we need a smaller SDK/API integration point so the SDK shell can still surface VS Code terminal affordances?

SDK shell path: what we gain and what we risk

What we gain:

• Good fit for one-shot, non-interactive commands where the model only needs stdout/stderr and an exit code.
• Less VS Code-owned PTY lifecycle, terminal emulation, shell integration, resize, and platform-specific shell behavior.
• Piped/non-TTY execution can avoid pager/color/control-sequence problems and is easier for the model to parse.
• Better parity with other SDK hosts if everyone uses the same default shell tool.
• Potentially simpler sandbox and telemetry ownership because execution stays in the SDK/runtime path.

What we risk losing:

• No real VS Code terminal to focus, reveal, scroll, copy from, or manually inspect.
• In particular, a spawned non-interactive shell may not load the same startup files as a user's integrated terminal. For example, zsh -c typically does not load .zshrc or .zprofile (.zprofile is login-shell behavior, .zshrc is interactive-shell behavior), and non-interactive bash -c does not load .bashrc unless configured through BASH_ENV. There would also be no shell integration script that runs.
• Shell execution may bypass AHP's first-class terminal resource model entirely: no terminal URI, no root terminal entry, no claim transfer, no terminal/input/terminal/data stream, no resize/control, and no command detection for the client to subscribe to.
• No direct user input path for prompts, REPLs, sudo/password-like flows, CLIs asking questions, or commands that need Ctrl+C.
• Shell/profile initialization may differ from the user's integrated terminal. A direct spawned shell may skip .zshrc, .zprofile, .bashrc, or tool-specific activation hooks unless the runtime explicitly asks for login/interactive shell behavior or injects an equivalent profile.
• Less user trust when commands are slow, hung, or producing important output somewhere the user cannot inspect.
• Accessibility risk: screen reader users may lose the familiar terminal surface and "terminal is awaiting input" workflow.
• Weaker recovery workflows: user cannot quickly type a fix into the same terminal, change Node versions, accept a prompt, or stop a process without spending another agent turn.
• No terminal command markers, shell integration readback, command reveal, or terminal-location telemetry equivalent unless we build another bridge.

Agent Host custom terminal path: what we gain and what it costs

What we gain:

• VS Code owns a PTY-backed terminal resource and can surface it in the terminal panel/editor.
• Tool output can include a terminal URI/session ID, enabling Focus Terminal, command reveal, and terminal revival.
• The agent can still get text output for the model while users can inspect the real terminal.
• The agent can send input to a running shell via write_*, while users can also type directly into the terminal for sensitive prompts.
• Long-running processes can remain visible and interruptible.
• The implementation can share workbench terminal concepts: shell integration, command markers, history suppression, prompt/input detection, and terminal telemetry.

What it costs:

• Catching up to runInTerminal fidelity or more contained one with proper shell integration: Improve Agent Host shell integration and command detection parity #317789
  • Output shown to the model must stay reliable even when the visible terminal has control sequences, alternate screen, progress bars, or huge scrollback.
  • write_*/stdin paths require careful safety and prompt guidance, especially around secrets and permission prompts.
• Sandbox behavior can diverge from the SDK shell path if not tested carefully.
• There is maintenance cost if the SDK shell keeps evolving and VS Code overrides it.

Code references behind this

• chat.agentHost.customTerminalTool.enabled is registered as an experimental/advanced setting that switches Copilot SDK sessions to the Agent Host terminal override.
  • src/vs/platform/agentHost/common/agentService.ts
  • src/vs/platform/agentHost/common/agentHostCustomizationConfig.ts
• When the custom terminal tool is enabled, Agent Host builds shell tools that override SDK built-ins and are backed by IAgentHostTerminalManager.
  • src/vs/platform/agentHost/node/copilot/copilotShellTools.ts
• The Agent Host custom tool currently provides primary shell execution plus read_*, write_*, shutdown, and list tools. The write_* path sends text to terminal stdin through terminalManager.sendText(...).
  • src/vs/platform/agentHost/node/copilot/copilotShellTools.ts
• Agent Host terminals inject VS Code shell integration, suppress shell history for agent commands, and set non-interactive env such as PAGER=, GIT_PAGER=, GH_PAGER=, and GIT_TERMINAL_PROMPT=0.
  • src/vs/platform/agentHost/node/agentHostTerminalManager.ts
• The workbench/local agent terminal tool has first-class UX for user input and direct terminal interaction.
  • src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/runInTerminalTool.ts
  • src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/sendToTerminalTool.ts
  • src/vs/workbench/contrib/terminalContrib/chatAgentTools/common/terminalSandboxService.ts
• The chat terminal UI can focus/reveal an existing terminal instance or open a terminal command URI.
  • src/vs/workbench/contrib/chat/browser/widget/chatContentParts/toolInvocationParts/chatTerminalToolProgressPart.ts
• Tests already encode the important distinction: SDK built-in bash can render a terminal pill, but has no AHP terminal session/URI; terminal content blocks do set terminalCommandUri.
  • src/vs/workbench/contrib/chat/test/browser/agentSessions/stateToProgressAdapter.test.ts
• Runtime context: Copilot CLI/runtime shell execution is now a direct child-process spawn path with stdio pipes rather than a PTY-backed terminal session. That path runs each command as a fresh child process, without terminal emulation, persistent PTY state, shell integration markers, or the same startup behavior as VS Code's integrated terminal.
• Protocol context: AHP defines terminals as first-class subscribable resources with their own URIs, root-state listing, ownership claims, client-dispatchable input, server-dispatched output, resize/control actions, and optional command detection.
  • https://microsoft.github.io/agent-host-protocol/guide/terminals.html

**Workflow tests**

• ✅ Baseline command execution: one-shot command returns correct stdout, stderr, and exit code.

  Manual test steps

  1. Ask the agent to run:

     printf 'stdout-line\n'; printf 'stderr-line\n' >&2; exit 7

  2. Expected with all paths: the agent can report stdout, stderr, and non-zero exit.
  3. Extra signal for terminal-backed paths: the command is visible in a terminal UI and can be focused/revealed.
  4. Extra signal for SDK shell path: output should be clean and parseable without requiring a terminal.

• ❌ Persistent shell state: a separate follow-up command can observe state set by a previous command.

  Manual test steps

  1. Ask the agent: "In the terminal, create .tmp/agent-host-terminal-test/subdir, cd into it, export AHP_TERMINAL_STATE=kept, and stop."
  2. In a separate follow-up request, ask: "Without cd'ing again, run pwd and printf '%s\n' "$AHP_TERMINAL_STATE"."
  3. Expected with a real persistent VS Code terminal: the second command sees the subdir cwd and env var.
  4. Expected risk with SDK shell path: if execution is process-spawn based rather than a persistent shell, cwd/env changes from a previous child process may not survive unless the runtime models them separately.
  5. Record whether the agent compensates by chaining commands or whether the terminal/session itself truly preserves state.
  Result

  Loss/missing persistent state:

  

• ❌ Python environment auto-activation: agent shell execution uses the same Python environment a user would get from VS Code terminal activation. Related: Make sure Python auto activation work with agent host #323164

  • Provide an explicit SDK API for built-in shell tool startup env/scripts github/copilot-sdk#1911 would make this possible.
  Manual test steps

  1. In a Python workspace, create a local venv:

     python3 -m venv .tmp/agent-host-python-venv

  2. Select that interpreter in VS Code.
  3. In a normal VS Code integrated terminal, verify the selected environment is auto-activated:

     python -c 'import os, sys; print(sys.executable); print(os.environ.get("VIRTUAL_ENV"))'

  4. Ask the agent to run the same command without explicitly activating the venv first.
  5. Expected with local/workbench terminal: python resolves to the selected venv and/or VIRTUAL_ENV is set the same way a user terminal would be.
  6. Expected with Agent Host custom terminal: terminal creation should preserve the VS Code/Python activation behavior, or at least expose the gap clearly.
  7. Expected risk with SDK shell path: the shell command may run as a child process without VS Code/Python terminal activation scripts, so python may resolve to the system interpreter and VIRTUAL_ENV may be empty.
  8. Record whether the agent notices the wrong interpreter before running tests/scripts, or whether it burns extra turns discovering the environment mismatch.

• 🟨Interactive prompt: the user or agent can answer a prompt in the same running process.

  Result

  Side note: things like sudo __ may take an extra step or two.

  Not sure if this is something we can avoid by tweaking the prompt for the model now we are in non-pty world for CLI SDK built-in shell tool. Need to test other interactive workflow such as with installs and multiple prompts, etc.

  Mixed results:

  Prompt with question:

  

  Other times, like sudo, we can't guide people to directly type into their terminal.

  
  Manual test steps

  1. Ask the agent to run:

     python3 -c 'name=input("Project name: "); print("created", name)'

  2. Expected with local/workbench terminal: the terminal shows the prompt; sendToTerminal or direct typing can answer it.
  3. Expected with Agent Host custom terminal: write_bash/write_powershell can answer it, and the user should also be able to focus/type in the terminal.
  4. Expected risk with SDK shell path: no VS Code terminal URI and no general stdin/write tool means the process may hang, time out, or require the agent to rerun with non-interactive flags.
  5. Accessibility signal: screen reader users should be told the terminal is awaiting input and have a Focus Terminal action.

• ❌Sensitive prompt: the user can type a secret directly into the terminal without sending it through the model.

  Result

  
  Manual test steps

  1. Ask the agent to run:

     python3 -c 'import getpass; v=getpass.getpass("Token: "); print("got-token", bool(v))'

  2. Type a dummy value such as test-token directly into the terminal. Do not send it through chat.
  3. Expected with terminal-backed paths: Focus Terminal is available and the secret stays in terminal input.
  4. Expected risk with SDK shell path: if there is no focused terminal/input channel, the agent may fail, ask in chat, or be unable to continue the original process.

• ❌ Progressive streaming on long-running command: a slow command makes progress visible and can be focused while running.

  Result

  

  Note: probably possible with SDK API.

  Manual test steps

  1. Ask the agent to run:

     python3 -u -c 'import time; [print(f"tick {i}") or time.sleep(1) for i in range(20)]'

  2. While it runs, verify whether the terminal panel/editor can be focused.
  3. Expected with terminal-backed paths: user can see streaming output in a terminal, reveal the command, and interrupt if needed.
  4. Expected with SDK shell path: the model may get partial output via read_*, but the user experience is inline text only unless VS Code adds a separate terminal-like surface.

• 🟨 Dev server / watcher: a server/watch process can keep running while remaining visible and stoppable.

  Result

  Stoppable when directly telling the agent to stop, but there is no way to ^c directly in the terminal or terminal inline preview.

  Manual test steps

  1. In a temporary folder, ask the agent to run:

     python3 -m http.server 8765

  2. Verify the user can see the live terminal output, focus the terminal, and stop it with Ctrl+C or the appropriate tool.
  3. Then ask the agent to make a follow-up file/read command while the server remains running.
  4. Expected with terminal-backed paths: the server remains visible in a terminal and user-controlled.
  5. Expected with SDK shell path: background execution may be represented by a shell ID and read_*/stop_*, but not by a VS Code terminal the user can inspect or type into.

• ❌ REPL / TUI / alternate screen: interactive programs are either supported or explicitly out of scope.

  Result

  

  Loss of feature, unable to interact with REPL.

  Manual test steps

  1. Ask the agent to run:

     python3 -i -c 'print("ready")'

  2. Try entering 1 + 1, then exit().
  3. Optional, if installed: try a harmless TUI such as top, less package.json, or an editor opening a scratch file.
  4. Expected with terminal-backed paths: the user can interact directly with the running program.
  5. Expected with SDK shell path: no PTY/terminal UI means REPL/TUI fidelity is missing or must be treated as unsupported.

• ✅ Large output and scrollback: users can inspect large output without losing terminal affordances.

  Manual test steps

  1. Ask the agent to run:

     python3 - <<'PY'
     for i in range(600):
       print(f"line {i:03d}")
     PY

  2. Expected with terminal-backed paths: user can scroll terminal output and reveal the command.
  3. Expected with SDK shell path: output may be compacted, truncated, or written to a file/log for model consumption; verify whether the user can inspect the same output ergonomically.
  4. Record whether inline output, terminal scrollback, and model-visible output agree.

• 🟨 Hanging command / cancellation / recovery: the user can tell what is happening and recover from a stuck process.

  Result

  No way to ^c to interrupt. The user needs to stop the entire session to stop the command, or try with a steering message.

  Manual test steps

  1. Ask the agent to run:

     python3 -u -c 'import time; print("started"); time.sleep(120)'

  2. Try: focus terminal, press Ctrl+C, then run echo recovered.
  3. Expected with terminal-backed paths: user can interrupt and continue in the same terminal.
  4. Expected with SDK shell path: stop_* may kill the process, but the user cannot directly interact with the terminal because there is no VS Code terminal instance.

• ⭐️❌ Terminal focus, reveal, hidden terminal, and telemetry: the chat UI exposes an obvious way to inspect shell execution.

  Result

  Loss of interactivity. Users are not able to view or interact with their Copilot terminal anymore.

  Manual test steps

  1. Run any command that streams output for several seconds.
  2. Collapse/hide the inline terminal output if possible.
  3. Verify whether there is a Focus Terminal / Show Terminal action.
  4. Verify whether the terminal opens in the expected location based on user settings.
  5. Expected with terminal-backed paths: terminalCommandUri or terminal session ID allows focus/reveal and telemetry.
  6. Expected with SDK shell path: tests currently expect no AHP terminal URI for built-in bash, so focus/reveal requires a different mechanism.

• ❌ Default shell, profile, and environment parity: agent shell behavior matches VS Code terminal expectations.

  Result

  Loss in user configuration from rc and profile scripts like zshrc and PowerShell profile.

  This would be possible once we get Provide an explicit SDK API for built-in shell tool startup env/scripts github/copilot-sdk#1911.

  Manual test steps

  1. Configure a non-default integrated terminal profile, or terminal.integrated.agentHostProfile.<os> if applicable.
  2. Ask the agent to run:

     printf 'SHELL=%s\n' "$SHELL"; printf 'PATH=%s\n' "$PATH"; printf 'TERM=%s\n' "$TERM"

  3. Expected with local/workbench terminal: behavior follows the VS Code terminal profile.
  4. Expected with Agent Host custom terminal: verify the pushed shell path is honored; note that current config describes only the shell executable path, not profile args/env.
  5. Expected with SDK shell path: verify whether SDK shell configuration matches VS Code user expectations or chooses its own shell/process flags.

• Accessibility-specific prompt flow: screen reader users can understand that input is needed and act without guessing.
  @meganrogge Ideally we should have screen readers test some of their usual workflows.

  Sanity test

  
  Manual test steps

  1. With screen reader mode enabled, run the ordinary input prompt scenario.
  2. Verify announcement/accessible view behavior for "terminal is awaiting input."
  3. Verify the user can focus the terminal and type directly.
  4. Expected with terminal-backed paths: terminal UI and chat elicitation/focus affordance should make the input state clear.
  5. Expected with SDK shell path: decide what equivalent accessibility surface exists if no VS Code terminal is attached.

Decision checklist

• Complete the workflow matrix above for local/workbench terminal, Agent Host custom terminal, and SDK shell path.
• Decide whether missing terminal visibility/input is acceptable product behavior or a blocker for VS Code Agent Host.
• Identify which failures are resolvable via additional support from SDK.
• Decide whether chat.agentHost.customTerminalTool.enabled should remain experimental, become the supported/default path, or be replaced by a smaller SDK terminal integration API.
• Define minimum parity before disabling/removing the custom path: Focus Terminal, direct user input, prompt/accessibility behavior, long-running command visibility, cancellation/recovery, and terminal telemetry.
• If we keep the SDK shell default, define explicit UX for "no real terminal exists" so users do not think VS Code lost terminal support accidentally.

VS Code's terminal value is broader than command execution: users expect visible output, direct intervention, accessibility affordances, terminal scrollback, command reveal, profile/shell fidelity, and a way to recover when the agent gets it wrong. This issue tracks whether Agent Host needs a VS Code-owned custom terminal tool to preserve those workflows, and what must be true if we rely on the SDK shell tool instead.

[anthonykim1]

@meganrogge There are still much to edit/add, but please feel free to add more workflow scenarios, or provide insights! Thanks