Plan 9, and what it means for an OS built on Felix

[el-bri-ga]

Plan 9 from Bell Labs is the closest ancestor of the operating system felixd is heading toward. This thread asks how much of its design to borrow, and where a log changes the answer. The long-term picture is in docs/design.md.

What Plan 9 got right

• Everything is a file, served over one protocol (9P). Devices, the network (/net), windows, and process control (/proc) are all file servers. One small protocol with about a dozen messages covers all of it.
• Per-process namespaces. Each process assembles its own view of the world with bind and mount. There is no global root that everyone shares, so isolation is mostly a matter of what you mount.
• Union directories. Several servers can be bound onto one path, with a defined order.
• Location doesn't matter. cpu runs your shell on another machine with your namespace imported, so a program can't tell where it runs.
• History is a directory. The Fossil file system archived daily snapshots to Venti (content-addressed, write-once), and /n/dump/2002/0315/... showed the file system as it was on that day.

How it maps onto Felix and felixd

Plan 9	Log-based OS
A file server	A workload that serves part of the namespace from its state
9P	Felix's wire protocol (streams, caches, conditional writes)
A file	A stream (append-only) or a cache entry (a value), or a workload folding edits into contents
Per-process namespace	What a workload's token can see and the names bound into its spec
Union directory	Merged inputs with a recorded order (#25)
cpu	Lease migration: a workload runs anywhere because its inputs and outputs are names, not addresses
/n/dump daily snapshots	Any offset, not just daily: fx at "2026-10-24 14:02" /path
Venti	Felix's log plus content-addressed modules and snapshots

Where the log changes things

• Plan 9 shares live state, and the log shares history. In Plan 9 a file server's crash loses whatever it held in memory. A felixd workload's crash loses nothing past the last record.
• 9P is synchronous RPC. Reading a file means a round trip to the server. A Felix stream is pushed to subscribers and replayable, which suits events better and random access worse.
• Files are mutable, and streams aren't. write at an offset in the middle of a file has no direct counterpart. It has to be a record that a workload folds in.

Where Plan 9 fell short

Plan 9 never displaced Unix, and some of the reasons apply directly to us.

• It asked people to leave everything behind. POSIX support (APE) was partial, drivers covered little hardware, and there was never a modern browser. Unix was good enough and ran the software people already had. For us: felixd runs beside Linux and native programs get a place on the path (step 5). An OS that requires a rewrite of everything won't be used.
• It was closed when it mattered. Plan 9 stayed proprietary through the 1990s and only became open source in 2000, under a license many people avoided until much later. By then Linux had the momentum. For us: MIT from day one, and design in the open.
• Distributed, but not fault tolerant. If a file server died, every mount of it hung or failed. Nothing replicated a server's state or moved it elsewhere. For us: this is exactly the gap the log fills, and the main reason felixd exists.
• Ad hoc text protocols behind "everything is a file". Each device and server invented its own ctl strings with no types or versioning, and graphics and signals (notes) strained the model. For us: record formats need schemas and negotiated versions, as Felix's wire protocol has. A workload's interface shouldn't become a pile of strings.
• 9P suffered over slow links. Programs issued one request at a time and waited for each reply, so anything far away felt slow. For us: streams push data, but every recorded effect is still a round trip through the log. That cost must be measured early, not discovered later.
• Namespaces were powerful and hard to reason about. Each process saw its own tree, so "which /net is this process using?" could be a real debugging question. For us: a workload's view of the namespace should be on the record and visible in ps and replay.
• Archives that never forget. Venti was write-once and kept everything, and Fossil, the file system in front of it, had reliability problems that led 9front to replace it. For us: history growth and redacting secrets are first-class problems, not a later concern.
• Its ideas spread in pieces. UTF-8, /proc-style interfaces and 9P all live on: 9P in Linux's v9fs, QEMU's shared folders and WSL. They spread because each could be adopted on its own. For us: felixd's parts, such as fenced output, replay and the namespace, should be useful on their own, inside systems people already run.

Questions

1. Effects as 9P. Design: effects after felixd #27 asks how workloads should reach the outside world. One answer is Plan 9's: a workload sees /net, /dev/time and so on as files, and every 9P request and response is recorded on the log. That gives a narrow, uniform effect interface that replays exactly. Is 9P the right shape for it, or is it too chatty?
2. A 9P gateway. Should felixd serve the namespace over 9P, so Linux can mount it (v9fs, plan9port) and cat, tail -f and ls work on streams and workloads? It would cost little and make the system explorable with ordinary tools.
3. Namespaces per workload. Should a workload's spec say "bind /svc/orders/in as my input" instead of naming streams directly, so that the same module runs against test or production data by remounting?
4. What to leave behind. Plan 9 never needed fencing, leases or determinism. Which of its ideas break once state has to be rebuilt from a log?
5. Avoiding its fate. Which of the shortfalls above is the biggest risk for us, and what would you do about it now?

Anyone who has run Plan 9 or 9front: what would you keep, and what did you find didn't age well?

[el-bri-ga]

The short version of this thread is now in the design as Lessons from Plan 9: four things to do as Plan 9 did and seven to avoid, each tied to a felixd decision. Disagreements belong here, and the design will follow where this thread lands.