Problem Statement
openshell-vm exec -- sh hangs with no visible output. Or openshell-vm exec -- top for example will fail saying there is no tty
Two issues combine to make interactive shell sessions unusable:
-
The shell doesn't know it's interactive. The exec agent inside the VM connects the subprocess to plain pipes instead of a pseudo-terminal (PTY). Shells like sh and bash detect this and skip the prompt, job control, and line editing. From the user's perspective, the terminal looks frozen.
-
Keystrokes aren't forwarded correctly. The host terminal stays in its normal "cooked" mode — it buffers input until Enter, interprets Ctrl-C locally, and echoes everything. This is a problem for a remote interactive session where the guest shell should handle all of that.
The result: openshell-vm exec works fine for non-interactive commands like echo hello, but is unusable for shells, top, vim, or anything that expects a real terminal.
Proposed Design
New wire frame: resize
Add a resize message to the host-to-guest protocol so the guest terminal matches the host window size:
{"type": "resize", "cols": 120, "rows": 40}
This sits alongside the existing stdin and stdin_close messages. Older agents that don't understand it simply ignore it.
Host side (Rust)
When the user's stdin is a terminal and the exec request has tty: true:
- Send the initial window size to the guest right after the exec request.
- Switch stdin to raw mode (every keystroke is forwarded immediately, no local echo, no local signal handling). The original terminal state is saved and restored automatically when the session ends — even on errors.
- While pumping stdin, check whether the terminal was resized and forward updates to the guest.
Non-interactive commands (openshell-vm exec -- echo hello) are unaffected — they skip all of this.
Guest side (Python)
The exec agent now dispatches based on the tty field in the request:
tty: false (default, unchanged): subprocess gets pipes, same behavior as before.
tty: true (new): allocates a PTY, sets TERM=xterm-256color, applies the initial window size before spawning the process, and forwards resize events during the session. The subprocess gets a real controlling terminal with job control and sign
al handling.
The existing pipe-mode code path also learned to silently ignore resize frames instead of treating them as errors. This means a new host binary works correctly even against an older guest agent that hasn't been updated yet.
What changes
| File |
What |
exec.rs |
Raw mode guard, terminal size query, resize frame type, updated exec and stdin pump logic, 19 unit tests |
openshell-vm-exec-agent.py |
PTY-based handler, window size helper, dispatcher refactor, pipe-mode backward compatibility |
What doesn't change
- Non-interactive / capture mode (
exec_capture) — always uses pipes
- The CLI (
main.rs) — already detects whether stdin is a terminal
- The socket connect path — no timeout changes
Alternatives Considered
-
Use crossterm or termion for terminal handling on the host. Decided against it — the nix crate already provides everything we need (tcgetattr, tcsetattr, cfmakeraw) and is already a dependency. No reason to add another crate for three function calls.
-
Signal-based resize detection (SIGWINCH handler with a self-pipe). More responsive than polling, but significantly more complex. Checking terminal size on each keystroke costs one syscall and covers the common case well. Can be added
later if needed.
-
Always use a PTY, even for non-interactive commands. Simpler, but would break openshell-vm exec -- echo hello where callers expect clean pipe-based stdout. The tty flag in the protocol already handles this distinction.
-
Wrap with script or socat inside the VM. Adds an external dependency and doesn't integrate with the existing frame protocol for resize handling.
Agent Investigation
- Traced the full exec path: CLI parses
exec subcommand, exec_running_vm() connects to the VM's vsock-bridged Unix socket, sends an ExecRequest, and pumps stdin/stdout.
- The
tty field already existed in ExecRequest but was completely ignored by the Python agent.
- The
nix crate (v0.29) with the term feature is already in the workspace — no new dependencies needed.
exec_capture() hardcodes tty: false, so programmatic use is safe.
- Found and fixed a buffered-data-loss bug in the initial dispatcher design: closing and recreating the socket file object after reading the request could drop bytes already buffered by the first reader.
Checklist
Problem Statement
openshell-vm exec -- shhangs with no visible output. Oropenshell-vm exec -- topfor example will fail saying there is no ttyTwo issues combine to make interactive shell sessions unusable:
The shell doesn't know it's interactive. The exec agent inside the VM connects the subprocess to plain pipes instead of a pseudo-terminal (PTY). Shells like
shandbashdetect this and skip the prompt, job control, and line editing. From the user's perspective, the terminal looks frozen.Keystrokes aren't forwarded correctly. The host terminal stays in its normal "cooked" mode — it buffers input until Enter, interprets Ctrl-C locally, and echoes everything. This is a problem for a remote interactive session where the guest shell should handle all of that.
The result:
openshell-vm execworks fine for non-interactive commands likeecho hello, but is unusable for shells,top,vim, or anything that expects a real terminal.Proposed Design
New wire frame:
resizeAdd a
resizemessage to the host-to-guest protocol so the guest terminal matches the host window size:{"type": "resize", "cols": 120, "rows": 40}This sits alongside the existing
stdinandstdin_closemessages. Older agents that don't understand it simply ignore it.Host side (Rust)
When the user's stdin is a terminal and the exec request has
tty: true:Non-interactive commands (
openshell-vm exec -- echo hello) are unaffected — they skip all of this.Guest side (Python)
The exec agent now dispatches based on the
ttyfield in the request:tty: false(default, unchanged): subprocess gets pipes, same behavior as before.tty: true(new): allocates a PTY, setsTERM=xterm-256color, applies the initial window size before spawning the process, and forwards resize events during the session. The subprocess gets a real controlling terminal with job control and signal handling.
The existing pipe-mode code path also learned to silently ignore
resizeframes instead of treating them as errors. This means a new host binary works correctly even against an older guest agent that hasn't been updated yet.What changes
exec.rsopenshell-vm-exec-agent.pyWhat doesn't change
exec_capture) — always uses pipesmain.rs) — already detects whether stdin is a terminalAlternatives Considered
Use
crosstermortermionfor terminal handling on the host. Decided against it — thenixcrate already provides everything we need (tcgetattr,tcsetattr,cfmakeraw) and is already a dependency. No reason to add another crate for three function calls.Signal-based resize detection (
SIGWINCHhandler with a self-pipe). More responsive than polling, but significantly more complex. Checking terminal size on each keystroke costs one syscall and covers the common case well. Can be addedlater if needed.
Always use a PTY, even for non-interactive commands. Simpler, but would break
openshell-vm exec -- echo hellowhere callers expect clean pipe-based stdout. Thettyflag in the protocol already handles this distinction.Wrap with
scriptorsocatinside the VM. Adds an external dependency and doesn't integrate with the existing frame protocol for resize handling.Agent Investigation
execsubcommand,exec_running_vm()connects to the VM's vsock-bridged Unix socket, sends anExecRequest, and pumps stdin/stdout.ttyfield already existed inExecRequestbut was completely ignored by the Python agent.nixcrate (v0.29) with thetermfeature is already in the workspace — no new dependencies needed.exec_capture()hardcodestty: false, so programmatic use is safe.Checklist