feat(backend): PTY-attach terminal streaming over /mux WebSocket

[Pritom14]

Closes #51

Summary

Ports the terminal-streaming surface from the legacy TypeScript repo to the Go backend. In the old stack a separate Node WebSocket process was required because Next.js cannot hijack the HTTP connection for an upgrade. Go's net/http upgrades in-process, so this lands as a single multiplexed /mux endpoint owned by the daemon, with the stream logic in a dedicated feature package.

The change is split into three reviewable commits: the feature package with its tests, the httpd transport that mounts it, and the daemon wiring.

What changed

New internal/terminal feature package (owns the workflow):

• protocol.go: the channel-tagged JSON wire protocol. Channels are terminal, subscribe, sessions, and system. Terminal payloads are base64 because raw PTY bytes are not guaranteed valid UTF-8.
• session.go: a single attached PTY session. Defines the PTYSource seam (supplies the attach command and liveness) and the ptyProcess/spawnFunc seam that lets fan-out, ring replay, and re-attach be tested without a real PTY, tmux, or network. Includes bounded re-attach with backoff for resilience across transient tmux drops.
• ring.go: a 50KB ring buffer that replays recent output to a client on subscribe so a reconnecting terminal is not blank.
• manager.go: the Manager that accepts connections, multiplexes channels, runs the write loop and heartbeat, and bridges the CDC broadcaster onto the sessions channel. Defines its EventSource and wsConn interfaces next to where they are consumed.
• pty_unix.go / pty_windows.go: the real spawn via creack/pty on Unix; a clear not-supported error on Windows for now.

internal/httpd transport (owns only upgrade and decoding):

• mux.go: mounts /mux, performs the WebSocket upgrade, sets a 1MiB read limit, and adapts coder/websocket to the manager's wsConn. It does not reach into the terminal internals. When the manager is nil the route is simply not mounted.
• server.go / router.go: thread the optional *terminal.Manager through New and NewRouter.

Daemon wiring (main.go):

• Constructs the tmux runtime adapter as the PTYSource, builds the terminal manager fed by the CDC broadcaster, and passes it into httpd.New. Raw PTY bytes never flow through the CDC change_log; only session-state events reach the sessions channel via the broadcaster.

How the flow looks

browser xterm.js
      |  WebSocket /mux  (channel-tagged JSON frames)
      v
httpd.muxHandler  (upgrade + 1MiB read limit + decode only)
      v
terminal.Manager.Serve  (multiplex channels, write loop, heartbeat)
      |                         \
      | terminal channel         \ sessions channel
      v                           v
terminal.session              cdc.Broadcaster.Subscribe
  PTYSource.AttachCommand       (session-state events only)
  tmux attach -t =<id>
      v
creack/pty  <-->  ring buffer (50KB, replayed on subscribe)

A client opens a terminal channel; the session attaches to the tmux pane, streams PTY bytes out base64-encoded, and replays the ring buffer to late subscribers. In parallel the sessions channel forwards CDC session-state events from the broadcaster so the UI can refresh without polling. If the pane drops, the session re-attaches with bounded backoff; on clean exit it reports exited and stops.

Testing

• The spawnFunc seam keeps the bulk of the suite hermetic: ring replay, fan-out, re-attach, protocol wire shape, and manager multiplexing all run without a real PTY or network.
• mux_test.go exercises the genuine upgrade plus wsjson plus Serve plus creack/pty path against a throwaway shell.
• session_integration_test.go drives a real tmux pane (TestSessionStreamsRealTmuxPane), guarded so it skips where tmux is absent.
• Full suite is green under go test -race ./...; go build ./... and go vet ./... are clean.

Notes

This was rebased onto the current main, which carries the trigger-driven CDC refactor. The sessions channel payload was updated to match the new cdc.Event shape (typed Type enum, ProjectID added, revision dropped).

[greptile-apps]

Greptile Summary

This PR ports the terminal-streaming surface from the legacy TypeScript/Node stack to Go, landing as a single multiplexed /mux WebSocket endpoint owned by the daemon. The implementation introduces a new internal/terminal package with PTY attachment, fan-out to multiple subscribers, a 50 KB replay ring buffer, and bounded re-attach logic, wired into internal/httpd via a thin upgrade adapter.

• internal/terminal owns the full workflow: channel-tagged JSON wire protocol (protocol.go), per-pane PTY session with re-attach resilience (session.go), ring buffer replay (ring.go), and the connection multiplexer (manager.go). Race conditions from previous reviews — replay/fanout ordering and the subscribe-to-assign window — are resolved via a deliver method that atomically appends to the ring and fans out under s.mu, and an exitFired guard that re-deletes a stale c.terms[id] entry if onExit fires before the assignment completes.
• internal/httpd/mux.go does only WebSocket upgrade + 1 MiB read-limit + coderConn adapter; all stream logic stays in internal/terminal.
• main.go wires the tmux runtime adapter and CDC broadcaster into the terminal manager and passes it to httpd.New; sessions carry the manager's context so Manager.Close() shuts them down cleanly on daemon exit.

Confidence Score: 5/5

Safe to merge. The two concurrency defects identified in previous rounds — replay/fanout duplicate delivery and the subscribe-to-assign stale-entry — are correctly closed by the deliver atomic method and the exitFired guard. No new correctness issues were found.

The central locking invariant (deliver serialises ring-append and fanout under s.mu; subscribe snapshots and replays under the same lock) is sound and well-tested including a 400-iteration race-cover test. The exitFired guard handles the late-exit window correctly. PTY teardown is idempotent via sync.Once. Daemon shutdown ordering (defer termMgr.Close() after HTTP server drains) is correct. The two suggestions are style/robustness improvements, not correctness blockers.

No files require special attention. The two inline suggestions on session.go (clear s.pty after close) and manager.go (collapse the double-lock in handleSubscribe) are low-risk improvements worth considering before merge.

Important Files Changed

Filename	Overview
backend/internal/terminal/session.go	Core PTY session lifecycle: attach, copyOut via deliver (atomic append+fanout under s.mu), ring replay under s.mu, exitFired guard for subscribe-to-assign window — all correctly serialised. Previous race conditions are fixed.
backend/internal/terminal/manager.go	Connection multiplexer: write loop, heartbeat, openTerminal subscribe-to-assign fix (exitFired + re-delete under c.mu), cleanup unsubscribes and closes cleanly. Stale entry bug is addressed correctly.
backend/internal/terminal/ring.go	50 KB ring buffer with snapshot-is-copy guarantee. append+snapshot always called under s.mu in production, so ringBuffer.mu is redundant but harmless. Append logic with tail-truncation is correct.
backend/internal/httpd/mux.go	Clean upgrade + coderConn adapter. InsecureSkipVerify noted in previous review comment (loopback-only intent); no other transport issues.
backend/internal/terminal/pty_unix.go	creackPTY wraps creack/pty; sync.Once on Close prevents double-Wait deadlock correctly. Kill before Wait for clean process teardown.
backend/main.go	Terminal manager constructed before httpd.New; defer termMgr.Close() runs after run() returns (after HTTP shutdown), so shutdown ordering is correct. CDC broadcaster wired to sessions channel cleanly.
backend/internal/terminal/manager_test.go	Thorough hermetic tests covering fan-out, replay, subscribe-to-assign window (400-iteration race test), heartbeat, and enqueue overflow. CDC forwarding and reopen-after-exit scenarios well covered.
backend/internal/terminal/session_test.go	Unit tests for fan-out, ring replay, write/resize passthrough, re-attach policy, and attach-command error path — all hermetic via fakeSpawner.
backend/internal/httpd/mux_test.go	Integration test exercises the real upgrade + wsjson + Serve + creack/pty path with a throwaway shell; system ping/pong also covered.

Sequence Diagram

sequenceDiagram
    participant Client as xterm.js (Browser)
    participant HTTP as httpd.muxHandler
    participant Mgr as terminal.Manager.Serve
    participant Sess as terminal.session.run
    participant PTY as creack/pty (tmux attach)
    participant Ring as ringBuffer (50 KB)
    participant CDC as cdc.Broadcaster

    Client->>HTTP: WS Upgrade /mux
    HTTP->>Mgr: Serve(ctx, coderConn)
    Mgr->>Mgr: spawn writeLoop + heartbeatLoop

    Client->>Mgr: "{ch:terminal, type:open, id:t1}"
    Mgr->>Sess: openSession(t1) → session.run(m.ctx)
    Sess->>PTY: defaultSpawn(argv) via spawnFunc seam
    Mgr->>Client: "{ch:terminal, type:opened}"
    Mgr->>Sess: subscribe(onData, onExit) [replay under s.mu]

    loop PTY output
        PTY-->>Sess: Read(buf)
        Sess->>Ring: deliver(chunk) [append + fanout under s.mu]
        Ring-->>Mgr: onData(chunk) → enqueue base64 frame
        Mgr-->>Client: "{ch:terminal, type:data, data:base64}"
    end

    Client->>Mgr: "{ch:terminal, type:data, data:base64 keystrokes}"
    Mgr->>Sess: session.write(raw)
    Sess->>PTY: Write(raw)

    Client->>Mgr: "{ch:subscribe}"
    Mgr->>CDC: events.Subscribe(onEvent)
    CDC-->>Mgr: onEvent(cdc.Event)
    Mgr-->>Client: "{ch:sessions, type:snapshot, session:{...}}"

    PTY-->>Sess: EOF / error → copyOut returns
    Sess->>Sess: shouldReattach? [IsAlive check + backoff]
    alt pane dead
        Sess->>Mgr: markExited → onExit callbacks
        Mgr->>Client: "{ch:terminal, type:exited}"
    else session alive
        Sess->>PTY: "re-attach (bounded to maxReattach=5)"
    end

Loading

_{Reviews (5): Last reviewed commit: "fix(terminal): guard subscribe-to-assign..." | Re-trigger Greptile}