Skip to content

How It Works

Jump to bottom
Ken Tobias edited this page Jun 22, 2026 · 5 revisions

How It Works

Overview

etr uses SSH only for the initial bootstrap — to start the server and exchange credentials securely. All subsequent terminal I/O flows over a persistent QUIC connection that survives network interruptions.

Connection lifecycle

1. etr SSHes to the remote host and starts etrs
2. etrs generates an ephemeral self-signed TLS certificate,
   binds a random QUIC port, prints "PORT <n> CERT <fingerprint>",
   and forks into the background
3. etr reads the port and certificate fingerprint from SSH stdout; SSH closes
4. etr opens a QUIC connection to the server, pinning the received certificate
5. etr sends SessionOpen (session ID + passkey) on the control stream;
   etrs responds with SessionAccept
6. Terminal I/O flows on the PTY stream; port-forwards on additional streams
7. On clean disconnect, etrs exits immediately

Why QUIC?

QUIC runs over UDP, so like raw UDP it isn't tied to a specific network path — etr can reconnect from a new IP or port without the OS discarding connection state. Unlike raw UDP, QUIC provides:

  • TLS 1.3 built in — all data is encrypted with no custom crypto needed
  • Reliable ordered delivery per stream — no dropped or reordered bytes reach the application
  • Multiplexed streams — PTY and each port-forward run independently; a stalled forward can't block the terminal
  • Congestion control — built in via QUIC's loss detection

Why not mosh?

mosh is the inspiration for the bootstrap model. etr differs in a few ways:

  • etr uses a proper ordered stream with replay and acknowledgement, not mosh's state-sync model
  • etr uses QUIC, which gives reliable delivery, TLS 1.3, and multiplexing for free
  • etr supports port forwarding (-L)

The bootstrap in detail

etr runs:

ssh -p <port> <host> etrs

and writes the bootstrap lines to the SSH process's stdin:

SESSION_ID/PASSKEY/TERM\n
[KEY=VALUE\n ...]        # env vars — LANG, LC_*, COLORTERM, TERM_PROGRAM forwarded automatically
[ETRCMD:<command>\n]     # optional: run a command instead of an interactive shell

etrs generates an ephemeral self-signed TLS certificate, binds a QUIC port, and prints PORT <n> CERT <cert_der_hex> to stdout, then forks:

  • Parent process: exits immediately → SSH sees the command return and closes cleanly
  • Child process: calls setsid(), redirects stdio to /dev/null, writes logs to ~/.local/state/etr/etrs.log, and runs the session

etr reads the port and certificate from SSH stdout, then opens a QUIC connection pinning exactly that certificate. No CA or PKI is involved — this is analogous to SSH host-key trust.

Security model

The passkey (a random 32-character string generated fresh each session) is exchanged over the SSH-encrypted bootstrap channel. Only a holder of the correct passkey can authenticate a SessionOpen message. The TLS certificate, also transmitted over SSH, prevents man-in-the-middle attacks on the QUIC connection. Together these provide the same security properties as SSH host-key pinning, without requiring the server to be pre-configured.

Reconnect

The client sends a heartbeat every 5 seconds. If 15 seconds pass without any packet:

  1. etr's per-connection tasks see QUIC errors and return
  2. etr opens a new QUIC connection to the same server address
  3. etr sends a new SessionOpen with the same session_id and passkey, plus its current last_received_seq watermarks
  4. The server matches on session_id + passkey, sends SessionAccept with its own watermarks
  5. Both sides replay any unacknowledged PTY data (seq > peer's watermark)

The server holds session state (shell process, PTY, stream history) for 30 minutes after the last packet. The reconnecting client may come from a different IP or port.

Port-forward streams are not replayed on reconnect — they are re-opened fresh by the client.

Stream layout

Every QUIC stream opened by the client begins with a 1-byte tag:

Tag Stream Content
0x01 Control SessionOpen / SessionAccept, then Heartbeat / TerminalResize / Disconnect
0x02 PTY Sequence-numbered raw chunks (terminal output server→client, stdin client→server)
0x03 Forward StreamOpen header then raw bytes (TCP) or UdpDatagram envelopes (UDP)

See PROTOCOL.md for the full wire format.

UDP forwarding

Each -L or -R UDP spec uses one shared QUIC stream for all datagrams, with every UdpDatagram envelope carrying the originating peer_addr and peer_port. Multiple concurrent UDP senders on the same forwarded port are fully supported:

  • -L (local → remote): the server-side forwarding function maintains one ephemeral UDP socket per unique source address. Each socket has a distinct OS-assigned port, so replies from the remote target return to the correct socket and are routed back to the right local sender.
  • -R (remote → local): the client-side handler applies the same per-sender socket model toward the local target. Reply envelopes carry the original external sender's address so the server routes them to the correct remote client.

Sender sockets are evicted after 30 seconds of inactivity. Suitable for DNS, STUN, and game-protocol use cases where multiple clients share a single forwarded port.

Login records (utmp/wtmp)

On Linux, etrs registers the session with the system login database via libutempter:

  • On connect: a USER_PROCESS entry is written to utmp/wtmp, so the session appears in who and last with the client IP address and via etr [PID].
  • On clean disconnect (shell exits or client sends Disconnect): a DEAD_PROCESS entry is written to mark the session as ended.
  • On signal (SIGTERM or SIGHUP): etrs writes the DEAD_PROCESS entry before exiting, so who/last entries are cleaned up even if the session is killed rather than ended normally.

On non-Linux platforms (e.g. macOS) the utmp calls are no-ops.

Logs

Process Log location
etr (client) ~/.local/state/etr/etr.log (when running interactively with -v)
etrs (server) ~/.local/state/etr/etrs.log (on the remote host)

Clone this wiki locally