Terminal stack teardown — drop tmux, keep xterm

[illegalcall]

Terminal stack teardown — drop tmux, keep xterm

A teardown of AO's terminal stack: today's implementation (tmux on macOS/Linux, conpty on Windows, xterm.js renderer) vs. direct PTY ownership, with concrete simplifications and fixes. Every claim is verified against code in this repo (not docs).

---

What's there today (verified)

• macOS/Linux: backend/internal/adapters/runtime/tmux/ — agents run inside tmux sessions; AO drives them via tmux attach, itself wrapped in a raw PTY by ptyexec/spawn_unix.go (which already uses creack/pty).
• Windows: backend/internal/adapters/runtime/conpty/ — a separate pty-host process (loopback TCP) + ptyregistry (restart recovery) + ring buffer. No tmux.
• Frontend: @xterm/xterm 5.5.0 + WebGL 0.19.0 (prefer WebGL, fall back to canvas — XtermTerminal.tsx).

The two layers (this is the key mental model)

• Multiplexer (tmux) = server-side; owns the PTY and produces bytes. Replaceable.
• Renderer (xterm.js) = client-side; draws those bytes. Standard — keep.

They're orthogonal. Replacing the multiplexer does not replace the renderer, and vice-versa.

Recommendation 1 — replace tmux with direct PTY

creack/pty (v1.1.24) is already a dependency and ptyexec already spawns a raw PTY — it just points it at tmux attach. Point it at the agent command instead, and add a ring buffer for scrollback (the conpty/ring.go pattern already in the repo).

Feature parity, with the benefit of switching:

Feature AO relies on today	tmux gives it via	Direct PTY (creack/pty)	Verdict	Benefit of switching
Full-screen agent TUI	tmux is a full terminal	raw PTY — the exact thing tmux wraps	same	Byte-perfect output; kills escape-sequence translation bugs (ESC[>84;0;0c flood #2094, cursor/repaint desync)
Scrollback	tmux scrollback	in-memory ring buffer	same	You own the buffer — bounded, deterministic truncation
Detach/reattach	tmux session persists	PTY master stays open + ring replays	same	One shared implementation across all three OSes
Multiple clients (desktop + mobile)	tmux multi-attach	terminal mux fans out	same	No tmux "smallest client" sizing surprises
Resize	tmux resize	pty.Setsize + SIGWINCH (already in ptyexec)	same	No tmux reflow layer; fewer repaint desyncs
Paste / bracketed paste	tmux paste-buffer	raw writes + conpty enterDelay	same	No paste-buffer abstraction; one shared input path
Kill process tree	tmux pane kill	process-tree.go + setsid	same	Drops the "pane pid == session id" hack and killSessionsByPID machinery
Liveness probes	probe tmux session	probe the process directly	better	Truthful probes — observes the real process, not a tmux session that can misreport
Survive daemon restart	tmux server outlives AO	child dies with daemon	lost (recoverable)	With sidecar + fd-handoff: persistence and zero-downtime upgrades

Cross-cutting benefits: drops the tmux binary as an install prerequisite; one runtime code path on all three OSes (today tmux and conpty are two divergent implementations); no persistent tmux server (our own tmux.go documents a footgun where the first CLI call auto-starts a server pinned to a now-deleted path); deterministic process model; byte fidelity becomes testable (raw bytes in = raw bytes out, impossible to assert through tmux).

The one cost, and how to recover it

tmux survives daemon restarts; an in-daemon PTY dies with it. AO already has the recovery shape on Windows (separate pty-host + registry + AO_RUNTIME_LAUNCH_ID reattach). Port that to macOS/Linux: a small PTY-owning sidecar over a Unix socket (0600) with fd-handoff, so even daemon upgrades don't drop running shells.

Renderer: keep xterm, fix the addon

Renderer bugs are separate from the multiplexer. If "garbled output / cursor jumps" is WebGL-related, the lever is the WebGL addon version (0.19.0 stable is the usual suspect — XtermTerminal.tsx is full of WebGL-atlas workarounds). Mitigations, cheapest first: force the canvas renderer (already the fallback), then patch/upgrade the WebGL addon. Replacing xterm itself is not worth it — it's the category-standard renderer.

What I'd do first

1. Drop tmux → direct PTY + ring buffer — biggest complexity win, deps already in tree.
2. Confirm the renderer is the bug by forcing canvas (loadRenderer fallback); if garbled output disappears, backport the WebGL texture-clamp patch (see the addendum comment below).

---

PTY lifecycle cleanup — accumulation is "everything around the kill path"

A comparable codebase hit the macOS PTY-master limit (509/511) after a handful of parallel workspaces and root-caused it to lifecycle gaps around an otherwise-correct per-terminal kill path. The findings map directly onto AO's runtime/reaper:

• No idle TTL, no session cap — a session lives forever unless its row is explicitly marked dead. Add an idle TTL + a max-session cap.
• Reaper only acts on delete/exit — closed-but-not-deleted sessions (or a missed dispose) leak forever. Reaper should also reap idle/stale sessions, not just deleted ones.
• Fire-and-forget dispose swallows errors — a transient failure silently leaks. Stamp disposeRequestedAt and have the reaper retry.
• App-quit kills nothing — a detached daemon + descendants survive. Make quit dispose (or reap) explicitly.
• Daemonized descendants escape the tree kill — double-fork + setsid (MCP servers, build daemons, watchers) reparent to pid 1 and get a fresh pgid, invisible to a ppid-walk + kill(-pgid). AO's own process-tree kill has the same escape hatch; add a kill-by-cwd (workspace dir) backstop, not just the ppid tree.
• Renderer "parked ≠ disconnected" — a hidden terminal still holds a live WS, an xterm instance, timers, and parses live output every frame; chatty hidden agents = invisible CPU drain. Park should mean disconnect (drop WS, stop parsing), and focus fan-out reconnects need jitter/cap (a global focus event reconnecting N terminals at once throttles the WS handshake).

The kill path itself is usually correct; accumulation comes from the lifecycle around it. Cheap wins: idle TTL + cap, kill-by-cwd backstop, and park = disconnect.

---

Terminal wheel/scroll handling: fractional lines + an escape hatch

A comparable terminal layer replaces xterm's stock wheel handling with a custom one that accumulates fractional lines across events (signed), handles DOM_DELTA_LINE/PAGE modes, and keeps a context key so a buffer/mode change resets the accumulator. The notable trick is the escape hatch: a localStorage flag (STOCK_WHEEL=1) reverts to stock xterm behavior, checked per wheel event rather than at install — because terminal instances are parked and reused across React mounts, an install-time check would need a full window reload. Cheap, targeted terminal-UX improvement with a zero-rebuild rollback switch.

---

Terminal mode replay: reattach must re-assert mode state

A comparable codebase hit a subtle terminal bug worth guarding against in AO's replay path: a kitty-aware TUI (Codex) emits \x1b[>7u once at startup to enable the kitty keyboard protocol, but the host's scrollback is a fixed 64 KiB FIFO — long conversations evict that enable byte. On renderer reload, a fresh xterm (kitty flags = 0) reattaches, the replay no longer contains the enable, and the TUI never re-pushes it — so Shift+Enter silently changes meaning mid-conversation. The same class is latent for bracketed paste, focus reporting, mouse tracking, app cursor, and cursor visibility.

The fix is VSCode's scaffolding with a corrected policy: run a headless xterm on the host, feed every PTY chunk to it alongside the FIFO, and on replay build a mode-state preamble from the headless state (kitty → \x1b[=N;1u, bracketed paste → ?2004h, focus → ?1004h, mouse → ?1000h/?1002h/?1003h) and send it before the FIFO bytes. AO's ring-buffer replay should do the same: persist terminal mode state (or re-derive it via a headless parser) and re-assert it on attach, instead of replaying raw bytes and hoping the enable sequence survived.

[illegalcall]

Part 1 addendum — the WebGL renderer has an unpatched texture-memory bug

Concrete, actionable finding for the "garbled output / cursor jumps" terminal bugs.

The bug. AO ships @xterm/addon-webgl 0.19.0 unpatched. In that build, the glyph-atlas max texture size is taken unclamped from the GPU:

this._deviceMaxTextureSize = this._gl.getParameter(this._gl.MAX_TEXTURE_SIZE)

On GPUs/drivers that report MAX_TEXTURE_SIZE = 16384, each glyph-atlas page can be allocated at 16384² RGBA ≈ 1 GiB. Theme changes, font changes, terminal splits, and reconnects each add pages, multiplying that.

The fix (proven in a comparable codebase that patches the newer addon line) is a three-line change:

1. clamp maxTextureSize to Math.min(4096, …) in the atlas / glyph renderer,
2. clamp _deviceMaxTextureSize to Math.min(4096, …) in the WebGL renderer,
3. zero canvas.width / canvas.height when an atlas page is removed, releasing the backing store immediately instead of waiting for GC.

This is the same class of bug AO's own XtermTerminal.tsx is already dancing around (the zero-sized-renderer Viewport crash, the WebGL atlas warm-up one-frame transient, the 600ms font-metric backstop), and it plausibly explains #3309 (smudged output + erratic cursor jumps) and #3803 (viewport disposal).

Actions, cheapest first:

1. Stopgap: force the canvas renderer (the loadRenderer fallback already exists) and see if the bugs disappear — that confirms the WebGL path as the cause.
2. Proper fix: backport the 3-line clamp + eager-free to 0.19.0 via a patchedDependencies entry, or upgrade to the addon line that already carries it.

This is the renderer half of the terminal stack — fully independent of the tmux → direct-PTY change above. Both halves can proceed separately.

[illegalcall]

After a second independent pass over current main and the upstream renderer sources, I would amend the recommendation rather than make “tmux → direct PTY + ring” the first implementation step.

The runtime/renderer split is right: tmux is replaceable, and xterm can remain the browser painter. A detached PTY host is also a reasonable destination. The part I no longer think is correct is the feature-parity claim for a raw replay ring.

The missing abstraction is a terminal authority

The mode-replay section at the end of the original post actually exposes the gap. AO's current Windows Ring keeps 1,000 newline-delimited chunks of raw output and omits the current partial line from Snapshot. A suffix of output bytes is not a terminal snapshot:

• it may start in the middle of an escape sequence;
• it does not describe the current normal/alternate screen, cursor, margins, modes, palette, title, or hyperlinks;
• replaying it at a different grid can produce a different screen;
• a short mode preamble repairs a few flags, but not the screen and scrollback state that those bytes produced.

There is another ownership problem: terminal queries. With direct PTY fan-out, every attached renderer can see DA/DSR queries and potentially answer them, while a deliberately disconnected/headless pane has no renderer to answer. Current AO deliberately avoids forwarding xterm's generic onData because it includes terminal-generated control replies that can corrupt the real PTY. Direct PTY alone therefore does not eliminate the suspected response/reconnect mechanism in #2094; it can move the ambiguity from tmux to N browser terminals.

I would introduce a TerminalAuthority boundary with these responsibilities:

1. A detached, versioned host owns the PTY/ConPTY and child process.
2. One headless VT model consumes every child byte, owns normal/alternate buffers, scrollback, cursor and mode state, and answers terminal queries exactly once even with zero viewers. @xterm/headless is one possible prototype, with explicit extensions for state a stock serializer does not cover (notably kitty); an embeddable Go parser would avoid adding a Node runtime to the daemon.
3. Attach is an atomic snapshot/subscription operation: {epoch, sequence, grid, snapshot} followed by updates after that sequence. Slow viewers get a bounded queue and resynchronize from a newer snapshot rather than blocking the PTY or accumulating unbounded bytes.
4. The authority owns input arbitration and one explicit resize policy. A PTY still has one grid; removing tmux does not remove the multi-viewer sizing decision. AO's current host chooses the largest single client by area, which is a policy worth preserving or deliberately replacing.
5. The host is registered by session + launch generation, and exposes separate facts for host alive, PTY child alive, and managed workload alive.

The browser can still use xterm. Initially the authority can encode a full ANSI repaint plus ordered ANSI updates; later the protocol could carry semantic cells/diffs to a custom painter without changing PTY ownership. This gives us a migration path rather than coupling the runtime rewrite to a renderer rewrite.

A detached host plus registry/socket is sufficient for daemon restart. FD handoff is only needed to replace the host itself, and it does not make a host crash recoverable unless another process already holds the master FD and the VT state is checkpointed/transferred. The wire contract can be shared across OSes, but Unix PTY, ConPTY, and process containment will remain platform-specific implementations.

Two smaller corrections to the parity table:

• AO does not currently use tmux paste-buffer; it uses chunked send-keys -l plus an enter delay.
• “Byte-perfect output” is not the right acceptance criterion. Different ANSI streams can produce the same screen, while replaying identical bytes from the wrong initial state can produce a different screen. The useful invariant is terminal-state convergence after attach.

Lifecycle: repair ownership, not “idle”

Persisting disposeRequestedAt and retrying idempotent cleanup is a good idea. I would not add a blanket idle TTL or make kill-by-CWD an ownership backstop:

• idle is a valid durable agent/shell state, not evidence that the user has abandoned it. A resource cap should reject a new session, evict only safely exited/terminated entries, or apply an explicit user policy—not silently terminate an idle worker.
• CWD is neither necessary nor sufficient proof of ownership: an unrelated user process can be inside the worktree, and an owned daemon can chdir away. Prefer launch-generation identity plus platform containment (for example cgroups on Linux and Job Objects on Windows), with process groups/registries and visible residual-process reporting where the OS cannot provide a hard container.
• The app-quit statement is stale on current main: an app-owned daemon now self-stops through the supervisor link, with a fallback group kill, while terminal sessions intentionally survive for adoption.
• “Park = disconnect” is a useful resource mode, not a universal rule. Retained terminals currently share one mux WebSocket, although each hidden xterm/parser/WebGL context still costs resources. The connection avoids replay/repaint on every route switch. I would keep a small measured hot set and disconnect an LRU cold set under a viewer/CPU/memory budget. Open PR fix(terminal): stabilize xterm viewport, session-switch render, colors, and activity state #4010's six-terminal LRU is a concrete version of that experiment; a canonical snapshot is what ultimately makes cold reconnect safe.

The wheel proposal is also largely already present: the current handler accumulates pixel deltas and handles line/page modes. The remaining useful refinement is to preserve fractional line-mode deltas too, reset the accumulator when the buffer/mouse-routing context changes, and optionally add a typed diagnostic flag; it is not a missing core behavior.

WebGL addendum: good experiment, different evidence

Forcing canvas as an A/B test is sound. The more specific texture claim needs qualification:

• Upstream still reads raw MAX_TEXTURE_SIZE in both WebglRenderer and GlyphRenderer, so a newer official line does not currently carry the proposed 4096 clamp/eager canvas release.
• Ordinary atlas pages start at 512 and grow through repeated four-page merges (there is a separate oversized-glyph path). A 16K RGBA canvas is indeed about 1 GiB, but that is a worst-case allocation, not the normal size of every atlas page.
• Xterm handle replacement can dispose queued viewport work #3803 already has a distinct evidenced cause: queued viewport work runs after an old xterm is disposed. It should not be counted as evidence for the atlas-size hypothesis.
• For bug(desktop): terminal panel shows smudged/garbled output and cursor jumps erratically during agent activity #3309, current upstream contains directly relevant fixes for shared-atlas garbling across terminals, atlas page overflow, and stale rendering after a shared atlas clear. Those are stronger candidates for a retained multi-terminal UI than the unmeasured 16K-page theory.

I would test 0.19.0 WebGL vs canvas vs a build/backport containing those upstream fixes, while recording atlas page count/dimensions, live xterm/WebGL-context count, context loss, renderer memory, and the exact output trace. A 4096 clamp can still be reasonable defense-in-depth, but it should not be labeled the confirmed fix before that matrix reproduces the failure.

Safer migration order

1. Define the authority protocol and terminal-state convergence tests; wrap the existing tmux and ConPTY paths behind it.
2. Add a Unix detached-host implementation with a real VT model. Do not use the current raw ring as the correctness layer.
3. Replay recorded traces and test startup with no viewer, attach mid-escape, alternate-screen state, mode bytes aged out, DA/DSR exactly once, resize, multiple client sizes, backpressure, renderer reload, daemon restart, and host failure.
4. Make the new host opt-in, compare resource usage and state snapshots, then make it default.
5. Remove tmux only after the new authority satisfies those invariants.

So I still agree that tmux can probably be removed. I disagree that deleting it first is the simplification boundary: build the terminal authority first, then tmux becomes just one adapter we can retire safely.

[illegalcall]

Part 3 — renderer landscape and canonical AO terminal direction

Date: 2026-08-17

This is the canonical continuation checkpoint for discussion #3991. It builds on the original proposal, WebGL addendum, and terminal-authority correction. It independently compares open-source terminal and agent-orchestration projects with AO and focuses on four boundaries that are often conflated: PTY/process ownership, authoritative terminal state, transport/replay, and final painting.

Executive answer

Verified: xterm.js integration is a real source of bugs and complexity in AO and in mature peer applications. AO, VS Code, Wave, and Tabby all carry renderer fallback, WebGL context-loss recovery, resize/fit coordination, and host-specific input workarounds. AO and Tabby also reach into xterm private internals. These are not hypothetical costs.

Correction: xterm.js is not the source of most terminal-system failure modes, and replacing it with a native embedded renderer would not eliminate PTY ownership, process lifetime, reconnect ordering, backpressure, resize authority, or terminal-query routing problems. A native renderer replaces part of the VT/input/rendering stack and introduces native-view, GPU, FFI, packaging, signing, accessibility, and per-platform integration work.

Recommendation for AO: do not make a big-bang xterm-to-native rewrite. First separate a durable terminal authority from interchangeable terminal painters:

agent process <-> persistent PTY owner
                    |
                    +-> canonical VT state and sole query responder
                    +-> atomic snapshot at sequence N + live events after N
                                      |
                             terminal-view contract
                        /             |              \
              xterm.js painter   Ghostty web     native experiment
              (baseline)         DOM/canvas      (macOS, later)

The first renderer bake-off should stay inside AO's existing Electron/web boundary: compare a current xterm baseline with wterm plus its Ghostty core and with ghostty-web. That isolates VT/painter behavior from native-overlay plumbing and preserves a path to Windows, Linux, and mobile. Test a full native painter later against the same recorded byte corpus, authority, input contract, and lifecycle stress suite. A native renderer should replace xterm only where it demonstrates measured wins large enough to justify platform-specific ownership. A separate semantic overview should serve agent monitoring; an exact interactive terminal should remain available for arbitrary TUIs.

What “embedded terminal” can mean

The phrase covers three materially different products:

1. A complete embedded widget/painter. xterm.js provides browser-side VT parsing, buffer state, selection/input behavior, accessibility hooks, and WebGL/canvas/DOM painting. cmux embeds a full Ghostty surface in an AppKit NSView; SwiftTerm provides AppKit/UIKit terminal views.
2. A headless parser and state engine. @xterm/headless, libghostty-vt, and libvterm maintain a grid, modes, cursor, and protocol state but do not by themselves give an Electron application a complete native UI surface. The host must still render cells and implement input, selection, IME, clipboard, accessibility, links, and graphics.
3. An externally painted terminal. Zellij, Herdr, Agent Deck, and NTM ultimately write ANSI to the user's existing terminal emulator. They can avoid shipping an embedded GPU painter because their product UI itself runs inside a terminal. AO cannot get an in-app desktop terminal from this approach without either launching an external terminal or adding a painter at the desktop boundary.

This distinction is the key to interpreting the projects below.

Revisions and method

All claims below are based on source or first-party documentation at these exact revisions. “Verified” means directly supported by those sources. “Inference for AO” is architectural analysis, not a claim made by the upstream project.

Project	Revision inspected
AO	8c7cdf0555eae21ff1acc33dcb27e0f16c1da8fc
cmux	7f9af0f90054ac4424e0e802da59e373626d6019
Ghostty	b97b17f06b1ffd694f80edd3df5dd2134a0bcb9e
electron-libghostty	5003e497f12c5046a015a0da798894662f164c6c
wterm	4a73024d9f9003972f9efa6fe1a9086d1c90417b
ghostty-web	1858a5947767a3e1c9e98dbf53b2ff87fedb2aab
Restty	7700b14a7643ba9240818209ef1e0aa90d83ad77
WezTerm	9c04f79f86649f76a8c978ff4b674f60297a6ec9
Arbor	d8d82b7eec6cba3682374875d8f13407c7181ef0
VS Code	3c7151d6e4a635fa3804f5113d399d70a915712c
Wave Terminal	a4447c1563b2df285ab89e76c82f91e1a1a49c1e
Tabby	14e2d60b9b6dee84a53c37f05eefeb803787de04
Zellij	85cc8b1fede608703a682631d41df5e51a8700b6
Herdr	51b7064ef0a02642393bab1d2eea0f4dbd8414d2
Superset	3842b447f6c2a96ef54fdb51c784c5fc4149e224
Vibe Kanban	4deb7eca8f381f7cbc1f9d15515a9ab8f8009053
Agent Deck	f0cec0a2ab3edee2c5cb0f992c89c6f13820432c
NTM	17d0a8af910de1628aff3d979c384ed6d16f04c6
SwiftTerm	c74d1e64d2079fe642b5e0538857b363d8335793
libvterm (Neovim fork)	934bc2fbf21800ac3458a499df8820ca5fb45fd3

Architecture matrix

Project	PTY/process owner	Authoritative terminal state	Attach/replay transport	Final painter
AO, Unix	tmux; one fresh attach PTY per viewer	tmux screen/scrollback/modes	raw attach bytes, JSON/base64 WebSocket	xterm.js WebGL/canvas
AO, Windows	detached ConPTY host	ConPTY plus a bounded raw-output ring; no canonical VT snapshot	raw ring snapshot then live bytes	xterm.js WebGL/canvas
cmux	normally Ghostty surface; manual-I/O mode allows an external owner	libghostty surface	in-process callbacks/output injection; best-effort app restore	native AppKit/Metal Ghostty surface
electron-libghostty	Ghostty surface in Electron main/native module	libghostty surface	N-API/IPC and native view callbacks	macOS AppKit/Metal overlay above Chromium
wterm	external PTY/backend	pluggable WASM core; built-in core or libghostty-vt	raw writes/WebSocket adapter	browser DOM rows
ghostty-web	external PTY/backend	libghostty-vt WASM	raw writes; xterm-compatible API	browser Canvas2D
Restty	external PTY/backend	libghostty-vt WASM; headless entry point also exposed	raw writes/WebSocket helpers	browser WebGPU with WebGL2 fallback
WezTerm	native mux/PTY domain	wezterm-term canonical state	sequence-numbered changed rows and stable row indices	native OpenGL/WebGPU frontend
Arbor	daemon-backed local PTY	Alacritty parser by default; experimental libghostty-vt	semantic terminal snapshots	native GPUI cell painter
VS Code	separate pty host using node-pty	headless xterm serializer for persistent processes	serialized ANSI replay plus live events	xterm.js, WebGL with DOM fallback
Wave	Go shell controller and PTY	live xterm in the browser; serialized xterm cache plus circular raw file	cached ANSI state plus raw tail/file events	xterm.js, WebGL with DOM fallback
Tabby	Electron main process using node-pty	renderer xterm	acknowledged raw IPC; PTY ID can be reattached	xterm.js WebGL/DOM
Zellij	Zellij server	server-side VTE parser and canonical grid	per-client character chunks encoded to ANSI	user's outer terminal
Herdr	Herdr server using portable-pty	vendored libghostty-vt, then a Ratatui cell frame	semantic cell frame or server-diffed ANSI	user's outer terminal
Superset	durable PTY daemon/subprocess	@xterm/headless plus explicit mode tracking	atomic serialized snapshot boundary, then live output	xterm.js
Vibe Kanban	one portable-pty session created for a WebSocket	renderer xterm only	base64 raw output; reconnect creates a new PTY	xterm.js
Agent Deck	tmux	tmux	normal attach for interaction; capture/pipe logs for preview	user's outer terminal
NTM	tmux	tmux for exact interaction; line/log state for monitoring	pipe-pane FIFO or capture-pane polling into sequenced events	user's outer terminal or monitoring UI

AO baseline: renderer and authority are already separate

Verified

On Unix, AO starts a fresh tmux -u -T RGB attach-session inside a real PTY for every viewer; tmux owns the durable pane and screen state, while the attach process emits a repaint (tmux attach, attach PTY). The terminal layer intentionally does not replay state because the runtime is expected to generate a full repaint for each attachment (attachment contract).

On Windows, a detached host owns ConPTY, keeps a bounded output ring, broadcasts live bytes, and atomically attaches a client to a ring snapshot before live fan-out (host ownership and sizing, snapshot boundary). This is durable process/transport ownership, but a suffix of raw output is not the same as a canonical screen and mode snapshot.

The desktop sends JSON/base64 output frames over a shared WebSocket mux (wire and socket pool, pooled connection). xterm then parses and paints those bytes.

AO's xterm component documents and implements concrete integration work:

• WebGL is preferred, canvas is the fallback, and renderer failure/context loss is handled explicitly (renderer selection).
• Font metrics, zero-size mounts, fit/resize ordering, hidden scrollbars, viewport selection, and disposal all need coordination; some workarounds read private _core state (private metric access, selection/scrollbar workarounds, fit and resize path).
• AO forwards physical key events rather than all onData output because xterm may generate terminal-query responses while replaying an attach repaint; forwarding those to the real PTY can corrupt the TUI (input boundary).

Inference for AO

The last point is not merely a rendering bug. It is an ownership ambiguity: which emulator is authorized to answer DA/DSR and related terminal queries, and whether a reply came from current output or historical repaint. A native painter would still need that contract. Swapping only xterm leaves tmux/ConPTY lifetime, raw-stream replay, backpressure, and resize arbitration unchanged.

AO is also testing an old xterm baseline

Verified: AO pins @xterm/xterm 5.5.0, canvas 0.7.0, and WebGL 0.19.0 (package manifest). On 2026-08-17, the npm registry's stable xterm tag is 6.0.0 and its beta tag is 6.1.0-beta.302 (registry). xterm 6.0 added DEC synchronized-output mode 2026 (implementation commit), which is specifically valuable for flicker-free full-screen TUI updates. Further viewport synchronization and concrete WebGL corruption/atlas fixes landed after the 6.0 tag (viewport follow-up, atlas-merge corruption, atlas overflow, stale shared atlas).

Inference for AO: the fair baseline is not today's 5.5 stack. Before concluding that xterm must go, AO should run the same corpus against stable 6.0 and a pinned 6.1 beta, with a non-WebGL fallback. That upgrade can plausibly fix flicker and corruption cases; it cannot fix AO's transport, PTY, replay, or resize-authority defects. A beta should be shipped only after AO's own trace and lifecycle gates, not because VS Code happens to track beta releases.

Full native embedding: cmux and SwiftTerm

cmux

Verified: cmux is a macOS-only Swift/AppKit application. Its README says it is not Electron and uses libghostty for GPU rendering (implementation summary, scope, platform). It creates a Ghostty surface by passing an AppKit NSView in ghostty_surface_config, and its terminal object explicitly owns the native surface lifecycle (surface creation, surface lifecycle).

The most relevant capability for AO is Ghostty manual I/O. In that mode Ghostty spawns no child process; input is delivered to a host callback, and externally owned output is injected with ghostty_surface_process_output (manual-I/O configuration). cmux also explicitly records whether the surface or a manual transport owns the process, PTY, and terminal protocol (I/O ownership). This is evidence that a native Ghostty painter can be separated from PTY ownership.

cmux does not prove transparent persistence of arbitrary child processes. Its restore contract saves layouts, directories, and best-effort scrollback but says arbitrary live process state is not checkpointed (restore contract).

Inference for AO: cmux is the strongest evidence that a libghostty painter could improve a native macOS terminal surface. It is not evidence of a drop-in cross-platform Electron replacement. Hosting an AppKit/Metal surface beside Chromium requires native view and lifecycle integration; Linux and Windows would need another complete surface or a separate AO renderer. cmux's source itself contains substantial native lifecycle, focus, input, resize, and renderer-recovery code, so the complexity moves rather than disappears.

electron-libghostty

Verified: electron-libghostty is the closest direct experiment to AO's desktop host. It creates a Ghostty CAMetalLayer surface in an AppKit NSView, places that native view above Electron's Chromium content, and synchronizes its rectangle, focus, occlusion, and lifecycle through the native module. Its documented integration measures a DOM element with getBoundingClientRect, multiplies by device scale, and sends every resize/scroll change over IPC (preview status and platform limits, DOM-to-overlay synchronization, surface and Metal layer). The project labels itself preview, macOS-only, and plans to move to Electron shared textures.

Inference for AO: this validates feasibility but also exposes the cost hidden by the phrase “embed Ghostty”: DOM/native z-order, clipping, scroll synchronization, focus, DPI, IPC, N-API ABI, signing, and crash ownership all become product code. It is a useful later macOS prototype, not the cleanest first test of whether xterm's painter is AO's problem.

Ghostty API scope

Verified: Ghostty describes libghostty-vt as the first cross-platform library layer: it parses sequences and maintains state on macOS, Linux, Windows, and WebAssembly, but its API signatures remain in flux (libghostty roadmap and current status). Separately, Ghostty's full C embedding entry point says it is currently not a general-purpose embedding API and is used to embed Ghostty within a macOS app (full embedding limitation).

Inference for AO: “use libghostty” currently presents two different risk profiles:

• use the complete macOS surface and accept a platform-specific native painter;
• use cross-platform libghostty-vt, then build and maintain AO's own painter and input/accessibility layer.

The latter can replace terminal-state parsing but does not replace xterm as a complete UI widget.

SwiftTerm

Verified: SwiftTerm offers a headless engine plus AppKit/UIKit terminal views and a convenience local-process view; its macOS PTY helper uses forkpty and resizes with ioctl (library surfaces, frontends, PTY implementation). The project advertises CoreText/Metal rendering and notes incomplete accessibility in its feature list (features).

Inference for AO: SwiftTerm is another credible Apple-native prototype candidate. AO should feed it externally owned terminal data instead of using a convenience view that owns the process; otherwise a view's lifetime would become the agent's lifetime. It does not solve AO's Windows/Linux painter requirement.

Ghostty-backed web painters: wterm, ghostty-web, and Restty

These projects matter more to AO's first renderer experiment than a native surface because they keep Electron, browser/mobile delivery, and the deployment boundary constant.

wterm

Verified: wterm separates a pluggable terminal core from a DOM painter. It offers a lightweight Zig/WASM core or an opt-in libghostty-vt core, renders only dirty rows on animation frames, implements synchronized-output mode 2026, and uses ordinary DOM text for selection, browser find, and screen-reader access (packages and features).

It is not renderer-bug-free. Its DOM painter manually handles wide cells and block glyphs; one source-level workaround pixel-snaps CSS gradients specifically because Claude Code's horizontal rule exposed inconsistent browser rounding (block-glyph workaround).

Inference for AO: wterm plus @wterm/ghostty is the strongest first alternative to benchmark. It removes WebGL contexts and xterm's JS parser while retaining AO's web/Electron/mobile shape, and it improves native selection/accessibility. The tradeoff is likely lower peak throughput and a different set of DOM layout/glyph bugs; neither claim should be accepted without AO's sustained-output and 20–50-session tests.

ghostty-web

Verified: ghostty-web exposes an xterm-compatible API around a Ghostty WASM state engine and a custom Canvas2D renderer, explicitly advertising migration by swapping the import (scope and API, usage). It implements its own selection, scrolling, link detection, input, dirty-row painting, and disposal above the VT core.

The repository contains a regression test for stale cells becoming visible after scroll growth under reset-heavy output (viewport-corruption regression). That is useful counter-evidence to “Ghostty core means rendering bugs disappear”: the core can be correct while the viewport/cache/painter around it is not.

Inference for AO: this is the lowest-friction parser-and-painter swap because its API resembles xterm. It avoids WebGL context and glyph-atlas failure classes, but Canvas2D still owns DPI, font metrics, selection hit testing, and redraw scheduling. It is a good bake-off candidate, not yet evidence for a production migration.

Restty

Verified: Restty combines libghostty-vt in WASM with TypeScript shaping and WebGPU, falling back to WebGL2. It exposes both an xterm-style wrapper and a DOM-free headless entry point, while describing itself as early-release software with APIs that may change (renderer and maturity, entry points).

Inference for AO: Restty is the most ambitious web-native candidate, but it rebuilds the same difficult GPU resource surface—glyph atlases, context/device lifecycle, shaders, fallbacks—that makes xterm WebGL costly. It may ultimately outperform both xterm and DOM painting, but its present maturity and GPU complexity make it a later benchmark rather than AO's first replacement candidate.

Browser-widget lineage: VS Code, Wave, and Tabby

VS Code

Verified: VS Code uses both node-pty and xterm packages (dependencies). PTYs live in a separate pty-host service with heartbeat/restart and attach/input/resize APIs (host lifecycle, process operations).

For persistent terminals, the host feeds output into @xterm/headless and @xterm/addon-serialize; a later client gets ANSI serialized from that headless state before live output (persistent process and serializer, serializer implementation). Its code also handles ConPTY DSR/CPR hangs when no renderer is attached, showing that terminal replies and attachment state remain explicit concerns (ConPTY query handling).

The client creates xterm and dynamically falls back from WebGL to DOM on context loss; it notes WebGL and DOM renderers can calculate different cell dimensions and therefore requests another resize (client construction, renderer fallback).

Inference for AO: VS Code is strong precedent for a durable authority plus disposable xterm painter. It also demonstrates that introducing a headless emulator does not remove mode/query/replay complexity. Using the same emulator family for headless snapshots and client rendering reduces semantic drift, but it couples authority behavior to xterm's parser and serializer coverage.

Wave Terminal

Verified: Wave's Go shell controller owns the PTY, continuously appends raw bytes to a circular terminal file, and separately handles input/resize (circular output file, PTY pump). The browser owns xterm, its serializer, and WebGL/DOM switching with context-loss fallback (xterm setup, renderer fallback).

For restore, the frontend loads a previously serialized xterm cache at its recorded byte offset, then consumes the remaining raw output tail. During idle periods it serializes current xterm state back to storage (cache plus tail, idle serialization).

Inference for AO: this is a practical cache design, not a single server-side terminal authority. If no renderer has consumed recent output, server-side semantic state and terminal replies are not inherently current. It illustrates how split ownership can work for an interactive desktop, but it is weaker for multi-viewer, remote, or crash-consistent AO attachments.

Tabby

Verified: Tabby's Electron main process wraps node-pty and uses an explicit acknowledgement queue to pause and resume PTY flow under renderer backpressure (PTY and acknowledgement queue, PTY ownership and IPC operations). A renderer proxy can reconnect to a surviving PTY by ID (renderer proxy).

Its xterm frontend implements flow control, WebGL recovery, input/resize, and private _core workarounds for scroll pinning, blank resize frames, and texture-upload flicker (frontend flow control and setup, renderer workarounds, WebGL recovery).

Inference for AO: Tabby directly validates the user's concern: browser-terminal integration can require brittle internals. It also validates a separate point: backpressure and PTY lifetime live outside the renderer and remain necessary after a renderer swap.

Server-owned semantic state: Zellij and Herdr

Zellij

Verified: a Zellij terminal pane owns a server-side grid/VTE parser and feeds every PTY byte into it. It pauses or intercepts some host queries and tracks pending input (pane state, PTY-byte handling). Rendering produces semantic character chunks from the grid; the output layer builds per-client changes and encodes them to ANSI (pane rendering, per-client output, ANSI serialization). Its server-to-client render payload is an ANSI string, which the client writes to stdout (IPC message, client output).

Herdr

Verified: Herdr owns its PTYs, feeds output into vendored libghostty-vt, composes terminal cells and application chrome into a Ratatui buffer, and transports either semantic cells or server-diffed ANSI to thin clients. Its external terminal performs final glyph rasterization (server workspace, semantic client renderer, ANSI client renderer).

Inference for AO

These designs move reconnect, inspection, and slow-client policy onto authoritative visual state. A later frame can safely supersede an unsent earlier frame because it represents a complete current result; arbitrary raw VT chunks cannot be dropped this way.

They do not show that AO can eliminate all painters. Zellij and Herdr can rely on an outer terminal because their primary clients are terminal applications. For AO, one of these must still be true:

• translate the server grid into ANSI and feed xterm/native terminal again;
• define a structured cell-diff protocol and build an AO cell painter;
• open the session in an external terminal and accept a split desktop experience.

A custom cell painter is a major product in its own right: Unicode width and shaping, bidi policy, cursor and selection, IME, keyboard protocols, hyperlinks, clipboard, accessibility, images, ligatures, font fallback, and GPU lifetime all become AO's responsibility.

Native semantic models: WezTerm and Arbor

WezTerm

Verified: WezTerm's native mux does not treat a renderer as the sole copy of terminal state. Its Pane interface exposes the current sequence number, the rows changed since an earlier sequence, stable row indices, cursor state, logical lines, and semantic zones (pane state contract). The GUI consumes this semantic state and paints it with WezTerm's native frontend.

Inference for AO: WezTerm is the strongest architectural reference for a state-authoritative mux with incremental painting. It is not a drop-in embeddable AO widget, but its sequence/change seam is closer to the correct long-term contract than forwarding an unbounded historical byte suffix to every painter.

Arbor

Verified: Arbor, another agent/worktree desktop, keeps daemon-backed terminal sessions, uses an Alacritty-derived emulator by default, offers experimental libghostty-vt, and gives its native GPUI view a semantic TerminalSnapshot. The inspected snapshot includes styled cell text, cursor, and a deliberately small mode set (app_cursor and alt_screen) (snapshot schema and engines, product behavior).

Inference for AO: Arbor demonstrates that a custom native cell view is viable when the product defines the supported terminal semantics. Its narrow snapshot is also a warning: a screen suitable for common agent output is not automatically full fidelity for arbitrary TUIs, graphics protocols, query replies, and accessibility. AO should state that product choice explicitly if it ever narrows the contract.

Agent orchestrators and terminal managers

Superset: durable PTY plus headless xterm authority

Verified: Superset's session object says it owns a PTY subprocess, a headless emulator, attached clients, and disk output. It feeds PTY output through the headless emulator before broadcasting and queues emulator work to prevent blocking the I/O path (session ownership and construction, headless setup and query responses, output pipeline, emulator queue).

The authority waits for a consistent emulator boundary, serializes a snapshot, and attaches live output after that boundary (snapshot boundary, attach). Its headless wrapper uses @xterm/headless, sends terminal-generated replies back to the PTY, serializes state, and separately parses selected DEC modes and OSC-7 (headless wrapper, query output, snapshot, extra mode parser).

The PTY daemon can hand inherited PTY master descriptors and snapshots to a replacement process (daemon handoff).

Inference for AO: this is the closest evidence for the proposed terminal-authority shape, not necessarily a code target. It shows both its value and its cost: consistent snapshots and a sole query responder become possible, but headless write scheduling, mode completeness, handoff, and renderer rehydration are substantial systems work. The extra hand-maintained mode parser also shows a semantic-drift risk when the underlying emulator does not expose every state needed by the product.

Vibe Kanban: simple PTY-per-WebSocket baseline

Verified: Vibe Kanban creates a portable-pty shell, stores it in a session map, and exposes write/resize/close methods (PTY service, operations). Its terminal WebSocket creates a new PTY, exchanges base64 raw output plus input/resize messages, and closes that PTY with the socket (WebSocket lifecycle). The web component is a small xterm/Fit/WebLinks wrapper (xterm view).

Inference for AO: this is an intentionally simpler endpoint, useful as a complexity floor. It does not provide AO's persistent agent terminal, canonical replay, multi-viewer, or daemon-restart requirements. A renderer that reconnects cannot restore a session when the backend creates a new shell per connection.

Agent Deck: tmux is the terminal authority

Verified: Agent Deck describes itself as a tmux-backed terminal session manager (README). It creates detached tmux sessions, configures them, and uses pipe-pane logs and capture-pane for observation (session creation, pipe log, capture). Interactive attach runs tmux attach-session under a PTY, copies it directly to the raw outer terminal, and forwards resize/input (interactive attach).

Inference for AO: this avoids embedded renderer bugs only because the user's existing terminal paints. It retains tmux as a dependency and gives the application less control over exact visual composition and accessibility. It is a credible optional “Open externally” path, not a full AO desktop replacement.

NTM: exact tmux session plus monitoring-grade event stream

Verified: NTM explicitly uses tmux as its execution control plane and exposes REST/SSE/WebSocket surfaces (README, capabilities). Its pane streamer prefers a tmux pipe-pane FIFO, batches output into line events, and falls back to polling capture-pane and hashing the full text (stream setup, FIFO streaming, capture fallback). The event hub adds sequence numbers, an in-memory ring, and optional SQLite persistence (event store, store and resume).

Inference for AO: NTM's events are monitoring-friendly lines, not a faithful interactive terminal frame. That is a useful product separation: AO's overview can use structured agent status/transcripts and avoid continuously painting every hidden terminal, while an exact terminal remains available on demand.

Parser/state library without a painter: libvterm

Verified: libvterm exposes terminal state properties, input/output and key/mouse APIs, state callbacks, and a screen-cell/callback API (state and input API, screen API). The inspected Neovim fork explicitly says Neovim currently bundles its own vterm rather than using this fork (fork status).

Inference for AO: like libghostty-vt, libvterm can underpin an authoritative grid or custom renderer, but it is not an embedded visual terminal. Adopting it means AO owns all painting and UI behavior above the cell API and must assess modern protocol coverage separately.

Which bugs does replacing xterm actually address?

Failure class	Can a native painter help?	Does it disappear?
WebGL context loss, Chromium canvas/atlas behavior, DOM measurement, private xterm renderer APIs	Yes, directly	xterm-specific cases do; native GPU/view lifecycle cases replace them
Font rasterization, shaping, emoji fallback, cursor painting, selection visuals	Potentially	No; responsibility moves to Ghostty/SwiftTerm/custom painter
Keyboard layouts, IME, Option/Alt, dead keys, mouse modes, clipboard, accessibility	Potentially	No; native event translation and host integration remain
PTY creation, child containment, crash adoption, process cleanup	No	No
Ordered streaming, replay boundary, slow clients, backpressure	No	No
Canonical normal/alternate buffers, modes, cursor, scrollback	Only if the chosen engine becomes the authority	No; it must be deliberately owned somewhere
DA/DSR/query responses and stale replay replies	Only if ownership is redesigned	No; exactly one live responder is still required
Multiple viewers and resize arbitration	No	No
Cross-platform packaging and updates	Usually worsens initially	No; native ABI/build/signing work is added

The user's diagnosis is therefore directionally correct at the painter/host-integration layer. It becomes incorrect if “xterm bug” is used as a catch-all for stream corruption, stale replay, terminal replies, process lifetime, or sizing races.

Strategy comparison

Keep xterm as both state engine and painter

Advantages:

• one cross-platform Electron implementation;
• mature VT/input/accessibility surface and existing AO integration;
• same family can run headlessly for server snapshots, as VS Code and Superset demonstrate;
• easiest path to browser/mobile viewers.

Costs:

• Chromium/WebGL/canvas lifetime and measurement bugs;
• hidden terminals consume parser and GPU resources unless parked or evicted;
• private APIs and browser event translation are fragile;
• server authority can drift from client state if only the painter parses output.

Ghostty VT core plus a web painter (wterm or ghostty-web)

Advantages:

• preserves AO's Electron, browser/mobile, and cross-platform delivery model;
• removes xterm's parser and WebGL-specific failure modes;
• wterm's DOM path gains native text selection/find/accessibility, while ghostty-web offers a low-friction xterm-like API;
• substantially cheaper to A/B than a native Electron overlay.

Costs:

• both projects are younger than xterm and own their own selection, input, sizing, viewport, and redraw bugs;
• DOM can cost more CPU/layout under output flood; Canvas2D makes selection and accessibility custom work;
• libghostty-vt API/build integration is still evolving;
• renderer replacement still does nothing to PTY, transport, replay, or resize authority.

Native embedded painter (libghostty surface or SwiftTerm)

Advantages:

• native text/GPU pipeline independent of Chromium;
• likely better local latency, font fidelity, and resource behavior if measured;
• Ghostty manual I/O preserves AO's external PTY authority.

Costs:

• complete Ghostty surface embedding is presently macOS-oriented; SwiftTerm's complete widgets are Apple-oriented;
• Electron/native child-view integration, focus, clipping, z-order, DPI, IME, accessibility, crashes, ABI, signing, and updater packaging become AO concerns;
• different platform painters can render or encode input differently;
• remote/mobile still need a web or semantic view.

Headless native VT engine plus AO-owned painter

Advantages:

• server can own canonical state, exact attach boundaries, inspection, and state-based backpressure;
• libghostty-vt is explicitly cross-platform.

Costs:

• this is the largest scope: AO becomes a terminal-renderer project;
• Unicode, graphics protocols, input, selection, accessibility, and GPU work are not supplied by the parser;
• engine API churn and FFI still apply.

External terminal delegation

Advantages:

• eliminates AO's embedded rasterizer and lets users choose Ghostty, Kitty, WezTerm, iTerm2, or another terminal;
• Agent Deck, NTM, Zellij, and Herdr show mature forms of this model.

Costs:

• breaks the integrated supervisor experience;
• window focus, navigation, previews, mobile access, and exact UI composition become harder;
• tmux or another persistent authority is still needed unless AO owns PTYs itself.

Semantic frames and structured agent views

Advantages:

• reconnect and slow-client coalescing operate on current state rather than an indispensable byte history;
• hidden sessions can show cheap status/transcript summaries instead of live GPU terminals;
• server-side search, observation, and mobile rendering improve.

Costs:

• a faithful cell protocol must cover width, styles, cursor, hyperlinks, graphics, modes, and versioning;
• a semantic agent transcript is not equivalent to an arbitrary terminal/TUI;
• if frames are re-encoded as ANSI into xterm, there are still two emulation/presentation stages.

Recommended AO direction

1. Measure and classify before replacing

For two weeks, tag every reproducible terminal failure by boundary:

• PTY/process lifecycle;
• authority/state/query response;
• transport/replay/backpressure;
• xterm parser/input;
• xterm WebGL/canvas/DOM painter;
• Electron host integration.

Record renderer, platform, GPU/context loss, terminal count, visibility/park state, bytes and writes per frame, attach sequence, resize epochs, and whether canvas fallback changes the result. This converts “xterm feels buggy” into a replacement budget and prevents a native rewrite from being judged against unrelated failures.

2. Define a deep terminal-authority interface

The daemon-side owner should eventually guarantee:

• one continuously drained PTY and launch identity;
• canonical normal/alternate buffers, scrollback, cursor, modes, title, and protocol state;
• one and only one responder for terminal-generated queries;
• atomic snapshot(sequence=N) followed by live changes strictly after N;
• explicit resize ownership and epoch;
• state-safe slow-client coalescing, or lossless ordered raw delivery;
• painter detach/crash without process loss.

The painter contract should receive state/output, report geometry, return user input, and expose capabilities. It should not own the agent process.

3. Build bounded prototypes, not one migration

Prototype A: headless authority with the existing xterm painter. On one opt-in runtime path, feed PTY output into a headless VT engine, handle query responses there, and attach with serialized state plus a sequence boundary. This tests the hardest correctness change while keeping the current visual surface. VS Code and Superset are relevant precedents.

Prototype B: web-painter bake-off. Behind one renderer adapter, feed identical recorded and live sessions into (1) current xterm 6.0 or a pinned 6.1 beta, (2) wterm's DOM painter with @wterm/ghostty, and (3) ghostty-web's Canvas2D painter. This isolates parser/painter correctness, resource use, accessibility, and latency without changing Electron/native integration or abandoning browser/mobile. Restty can join later when its API stabilizes.

Prototype C: macOS Ghostty manual-I/O painter. Only after the authority and adapter exist, feed that same contract into a native Ghostty surface and route Ghostty input callbacks back through the AO input path. Do not let the surface spawn or own the agent. Keep the winning web painter as the Windows/Linux/mobile and recovery path.

Do not combine authority replacement, binary transport, and native painter in one experiment; otherwise failures cannot be localized.

4. Add a semantic overview beside, not instead of, the exact terminal

Use agent hooks, structured events, process facts, and optionally a server-side screen model to render cheap waiting/running/error summaries. Park or evict hidden exact painters under a measured budget, and reconstruct them from authoritative snapshots when selected. NTM's separation between line events and exact tmux interaction is the useful idea here.

Acceptance tests and decision gates

Shared replay corpus

Record real PTY byte streams and replay them into xterm, a headless authority, and the native candidate. Compare final grid, scrollback, cursor, modes, title, and generated replies—not byte-for-byte renderer output. Include:

• normal and alternate screens;
• bracketed paste, focus reporting, mouse modes, application cursor/keypad;
• DA/DSR/CPR and replies during fresh output versus historical replay;
• Kitty keyboard protocol and common agent TUIs;
• split escape sequences and sustained output flood;
• combining marks, wide glyphs, emoji, ligatures, font fallback, and ambiguous-width characters;
• OSC titles, hyperlinks, clipboard policy, and supported graphics.

Lifecycle stress

• 20–50 sessions with rapid mount, park, unpark, and disposal;
• app/daemon restart, painter crash, stale registry, sidecar crash, and child exit;
• sleep/wake, display hot-plug, DPI and font changes;
• forced WebGL context loss and native GPU-surface loss;
• reconnect at arbitrary sequence boundaries and slow-client recovery;
• simultaneous desktop/mobile viewers with resize contention.

Input/accessibility matrix

• IME composition, dead keys, non-US layouts, Option/Alt/Meta, function keys;
• bracketed and large paste, clipboard permissions, mouse protocols, drag selection;
• screen reader navigation, contrast, cursor announcements, and keyboard-only use.

Resource and product gates

Measure per-visible and per-parked-session RSS, idle CPU, GPU contexts/memory, output-flood latency, first-paint/reconnect latency, binary size, crash isolation, and packaging complexity. Review licenses and API/version stability before selecting an engine.

A native renderer should ship broadly only if it materially beats the stabilized xterm path on reproducible reliability, fidelity, and resource tests and AO accepts the per-platform maintenance surface. Otherwise, keep xterm, fix authority/transport bugs at their actual layer, cap or park hidden painters, and offer native/external modes as optional paths.

Bottom line

There is no single project whose terminal architecture AO should copy:

• cmux is the best evidence for high-quality native Ghostty painting and a useful manual-I/O seam, but it is macOS-specific;
• wterm and ghostty-web are the cleanest first tests of a Ghostty-backed renderer inside AO's existing web boundary; each already demonstrates that custom DOM/canvas painting has its own correctness work;
• Restty is a promising but early Ghostty/WebGPU stack whose GPU-resource complexity overlaps the class AO is trying to escape;
• WezTerm is the strongest reference for a sequence-aware semantic pane contract, while Arbor shows both the viability and fidelity limits of a custom native cell painter;
• VS Code and Superset are the strongest evidence for headless canonical state plus a disposable xterm painter;
• Zellij and Herdr show the benefits of server-owned semantic state, but rely on an external terminal for final painting;
• Wave and Tabby expose both the practicality and the integration costs of xterm in Electron-style applications;
• Agent Deck and NTM show that tmux plus an outer terminal remains a valid control-plane choice, especially when monitoring and exact interaction are separated;
• Vibe Kanban shows the low-complexity raw PTY/WebSocket/xterm baseline, but not AO-grade persistence.

The globally stronger architecture is a hybrid: persistent PTY authority, canonical state and sequence-safe attach, a painter-independent contract, an upgraded xterm baseline, a Ghostty-backed DOM/Canvas bake-off, a later measured native macOS experiment, and structured agent views that avoid painting hidden terminals unnecessarily.

Continuation checkpoint

Future work on AO's terminal stack should start from these carried-forward conclusions unless new measurements invalidate one of them:

1. Do not equate a raw replay ring with terminal state. The correctness boundary is an authoritative VT state plus an atomic snapshot/sequence contract.
2. Do not let viewers answer child terminal queries. Exactly one authority answers DA/DSR and related queries, including when no viewer is attached.
3. Do not couple process lifetime to a painter. The PTY owner survives renderer mount, disposal, crash, park, and replacement.
4. Do not attribute every visible defect to xterm. Classify it as PTY/process, authority/query, transport/replay, parser/input, painter, or Electron integration first.
5. Do not compare a new renderer only against AO's xterm 5.5 stack. The control is a tested current xterm line with synchronized output and relevant fixes.
6. Test Ghostty in the web boundary before adopting a native overlay. wterm plus @wterm/ghostty and ghostty-web isolate the renderer hypothesis without adding platform divergence; native Ghostty manual-I/O is a later macOS experiment.
7. Keep exact terminals and agent overview views separate. Structured status/transcript views should avoid painting hidden terminals, while arbitrary TUIs remain available through an exact terminal.

To reopen one of these decisions, add a comment that names the numbered conclusion, supplies a reproducer or measurement, identifies the affected layer, and records the AO revision and renderer/runtime versions. That keeps the discussion cumulative rather than restarting from product analogies or symptoms.