Millhouse

Millhouse is the human-interactive surface around the frostyard mill: it supervises mill runs across projects so that a run's lifecycle — start, liveness, phase, stop — is owned by one place instead of being hand-scripted per run. Its core rule is that every process operation targets a pid/pgid recorded in a registry (never a pattern match, never a global "stop everything" verb), and that a run is considered alive only when its recorded PID exists and /proc/<pid>/cwd still resolves to its recorded worktree. See DESIGN.md for the product design, the motivating incidents, and the substrate probe results these packages preserve.

Scope of this repository today: the core supervision library, the HTTP control plane over it, and the millhouse binary — the Go packages listed under Packages. millhouse serve runs the daemon, and spawn, list, status and stop drive it over that API. Anything else described in DESIGN.md is design, not code.

Platform: Linux only, deliberately. /proc, setsid, and process groups are the substrate, not an implementation detail to be abstracted away.

Quickstart

Requires Go 1.26.

make build   # compile all packages; the daemon lands in build/millhouse
make test    # go test ./...
make ci      # mirrors CI exactly: tidy, vet, gofmt, test, race, build

make ci is the single source of truth for what CI checks — if it is green locally, CI is green.

Run the daemon, then drive it:

build/millhouse serve &                     # binds 127.0.0.1:7777, logs to stdout
id=$(build/millhouse spawn --project myproj --source local \
       --worktree /path/to/worktree -- sleep 300)
build/millhouse list                        # ID PROJECT SOURCE PID PHASE GATE STATE
build/millhouse status "$id"                # the same run as indented JSON
build/millhouse stop "$id"                  # stopped <id>

Stopping the daemon does not stop the runs: they are spawned detached, and a restarted daemon re-attaches to them from the registry.

⚠ There is no authentication. The daemon binds 127.0.0.1:7777 by default, and that default is the only thing protecting it. Anyone who can reach the address it is listening on can start processes on this machine, read their logs, and kill them. A non-loopback --addr — 0.0.0.0:7777, a LAN address, a container's external interface — is an explicit operator choice to publish that capability to the network, and nothing in this phase will ask for a credential. Put it behind an SSH tunnel or a reverse proxy that does authenticate, or leave it on loopback.

HTTP API

Seven endpoints, all served by millhouse serve. Every JSON error is the object {"error":"<message>"}; <run> below is the run document described under State directory. A request to a known path with the wrong method is answered 405 by the router itself.

Method and path	Success	Errors
GET /healthz	200 {"ok":true}	none — it reads neither the registry nor /proc, so a state-dir problem cannot make it fail
GET /runs	200 [<run>, …], started_at ascending (ties broken by id); [] when empty	none
GET /runs/{id}	200 <run>	404 no such run "<id>"
POST /runs — body {"project":…,"source":…,"worktree":…,"cmd":[…]}	201 <run> (the created run, as recorded)	400 malformed JSON, any of the four fields absent or empty, an empty argv[0], or a worktree that is not an existing directory; 500 any other spawn failure
POST /runs/{id}/stop	200 {"id":"<id>","escalated":<bool>} — escalated says whether SIGTERM had to be followed by SIGKILL	404 no such run "<id>"; 409 run <id> is not alive — nothing is signalled in either case; 500 when the run was alive but signalling its recorded process group failed
GET /runs/{id}/log?tail=N	200 text/plain, the last N bytes of that run's log (tail defaults to 4096, is clamped to 1 MiB, and a shorter log comes back whole)	400 invalid tail for a non-positive or unparseable N; 404 no such run "<id>", or run <id> has no log file yet; 500 when the log file exists but cannot be read. This endpoint's errors are text/plain, not the JSON error document
GET /runs/{id}/events?source=journal|conductor&tail=N	200 application/x-ndjson, the last N events of that stream as the producer's own JSONL lines, verbatim, oldest first and newest last (tail counts events, defaults to 50 and is clamped to 1000; conductor events are ordered by each event's own timestamp, never by filename). A known run that has produced no events yet is 200 with an empty body, not a 404	400 invalid source when source is absent or is anything but journal/conductor; 400 invalid tail for a non-positive or unparseable N; 404 no such run "<id>"; 500 when a stream's files exist but cannot be read. This endpoint's errors are text/plain, not the JSON error document

alive, phase, gate_pending, last_event_at and schema are recomputed for every response — from /proc, from the worktree's journal and .mill/meta.json, and from the run's conductor event log — and are never served as read from disk. POST /runs passes cmd to execve as a list; no shell is involved at any point, and no endpoint accepts a pattern, a glob or an "all" selector.

State directory

millhouse serve keeps everything in one directory, resolved from --state-dir, else $MILLHOUSE_DIR, else ~/.millhouse:

~/.millhouse/
├── registry.json        # every run record; the single source of truth for
│                        # what may be signalled. Written atomically
│                        # (temp file + rename), never appended to.
├── engine/
│   └── <version>/            # the three mill engine files this binary
│       ├── mill.yaml         # embeds, written out at `serve` startup;
│       ├── mill-copilot.yaml # <version> is read from mill_state.py's own
│       └── mill_state.py     # ENGINE_VERSION. Nothing reads them yet.
├── logs/
│   └── <run-id>.log     # that run's stdout and stderr, as the process wrote
│                        # them; served by GET /runs/{id}/log
└── runs/
    └── <run-id>/
        └── tmp/         # that run's private $TMPDIR, created before the
                         # process starts; conductor's event log lands in
                         # tmp/conductor/*.events.jsonl

Every run is started with TMPDIR set to its own runs/<run-id>/tmp — the daemon's environment plus that one variable — so the scratch files its tools write, conductor's event log above all, land inside the state dir keyed by run id instead of in a shared /tmp that is swept out from under them.

Nothing under runs/<run-id>/ is deleted by this phase: not when a run exits, not when a spawn fails after the directory was created (a failed spawn still gives its registry reservation back and removes its log file, but leaves that directory and anything the short-lived process wrote into it), and not on any sweep, age check or size cap — there is none. Retention is a later concern; today those directories, and the log files beside them, accumulate until an operator removes them.

engine/<version>/ holds the three mill engine files this binary carries embedded — mill.yaml, mill-copilot.yaml and mill_state.py — written out once at serve startup, before the listener is opened; if they cannot be written the daemon exits non-zero with a one-line message rather than serving from a half-written directory. Files already present with the right content and mode are left untouched; anything absent, edited or left with the wrong mode is rewritten atomically, and a path occupied by something that is not a regular file is an error rather than something to replace. Nothing under engine/ is deleted by this phase — not on startup, not on shutdown, and not on any sweep — so upgrading the binary adds a directory beside the old ones and every version materialized on this machine accumulates until an operator removes them. Retention is a later concern, exactly as for runs/<run-id>/ above. Nothing reads these files yet: materializing them is all this phase does with them.

registry.json is a JSON object of run id → run document:

{
  "9f3a1c2b": {
    "id": "9f3a1c2b",
    "project": "myproj",
    "source": "local",
    "worktree": "/path/to/worktree",
    "cmd": ["sleep", "300"],
    "pid": 40127,
    "pgid": 40127,
    "started_at": "2026-07-24T11:02:15.481Z",
    "exit_observed_at": "2026-07-24T11:07:15.502Z",
    "alive": false,
    "phase": "gate_chunk",
    "gate_pending": false,
    "schema": ""
  }
}

The value of each key is the <run> document the API serves.

exit_observed_at is absent while a run is alive. alive and phase are persisted as advisory snapshots for a human reading the file — no code path trusts them from disk.

gate_pending (bool), last_event_at (RFC3339 UTC) and schema (string) are advisory snapshots on the same terms, derived from files outside the registry: gate_pending and last_event_at from the run's conductor event log under runs/<id>/tmp, schema from the worktree's .mill/meta.json. last_event_at is omitted while empty (empty means "no event with a usable timestamp", which today includes every run with no conductor events, since the mill's journal lines carry no timestamp); the other two are always present, because false and "" are their real values for "no gate pending" and "no schema recorded".

All five derived fields are recomputed together on every read (GET /runs, GET /runs/{id}) and on every poller tick, for every run and regardless of liveness — the files they come from outlive the process, so a stopped run keeps serving what its journal, event log and meta.json still say. What was stored is never returned: a value that reached registry.json some other way is overwritten by the recompute, not passed through.

A run whose .mill/meta.json reports a schema other than 1 gets "schema?" as its phase, while schema keeps reporting the version verbatim. That is a warning about how far to trust everything else derived from that run's files — never an error, and never a reason to stop tracking, listing or stopping the run.

Packages

• internal/run — the run model and the registry that persists it.

  • Run — one supervised run (id, project, source, worktree, command argv, pid/pgid, start/exit times) and its persisted JSON shape.
  • Registry — run records keyed by run ID under an injected state directory, written atomically (temp file + rename); LogPath derives a run's log file from its ID, and RunTmpPath its per-run temporary directory (runs/<id>/tmp). Both are pure path functions — they take no lock, touch no disk, and create nothing.
  • Alive / Phase / GatePending / LastEventAt / SchemaOf — derived state, recomputed on every read rather than trusted from disk: liveness is pid and /proc/<pid>/cwd matching the recorded worktree; phase is the last event in the worktree's .mill/journal.jsonl; the last three read the run's conductor event log and .mill/meta.json through internal/events. Only Alive consults the process; the rest are functions of files that outlive it, so they are recomputed for stopped runs too. Each degrades to its safe zero value (false, "") when a source cannot be read — this is the read path, which never logs and never fails a caller over a file another process is writing.
  • The one judgement made on top of them: a schema that is neither "" nor "1" replaces the run's phase with "schema?", leaving schema itself verbatim. It is warn-only — no error, no dropped run.

• internal/events — read-only parsers for the two event streams a mill run produces: the mill's semantic journal (<worktree>/.mill/journal.jsonl, lines keyed event) and conductor's execution event log (<run-tmp>/conductor/*.events.jsonl, lines keyed type, timestamp, data). A Scanner takes both paths as values and re-reads them on every call — nothing cached, no read offsets persisted, exactly like Phase.

  • Each source has one parsed accessor — JournalEvents, ConductorEvents — which alone decides which lines survive and in what order, plus a raw-line accessor — JournalLines, ConductorLines — derived from it. The raw text is never filtered or ordered by a second implementation, so it cannot disagree with the parsed events.
  • Journal order is file order: one append-only file, so it is already chronological (and today's journal lines carry no timestamp at all). Conductor events are sorted by their own timestamp field, never by filename — a conductor filename embeds a workflow name as well as a time, so filenames do not sort chronologically across workflows.
  • Malformed lines are skipped, never fatal, and dropped from the raw view as well as the parsed one; a missing worktree, run-tmp, directory or file reads as empty rather than as an error.
  • Three derived signals read on top of those accessors: GatePending (is the newest gate-type conductor event a gate_presented rather than a gate_resolved?), LastEventAt (the newest conductor timestamp as a UTC time, with an ok flag so "no events yet" stays distinct from "an event at the epoch"; journal lines have no timestamp, so they never contribute), and Schema (the schema field of <worktree>/.mill/meta.json, which the engine writes as a JSON number, normalized to its canonical string form — both 1 and "1" read as "1"; an absent file or field reads as ""). Schema only normalizes the value; deciding what a given schema means for a run belongs to the layer that owns a run's phase.
  • internal/run is its only consumer today: the registry's recompute path reads these three signals for every run. Nothing above internal/run imports it yet.

• internal/supervisor — the process operations over that model.

  • Spawn — starts a run detached (setsid, its own session and pgid), exec'ing argv directly, with output to the registry's log path and TMPDIR set to the registry's RunTmpPath for that run (this process's environment plus that one appended variable — constructing a child's environment, not reading configuration out of one). The run id is reserved first — a pid=0 registry entry, plus the log file claimed with O_CREATE|O_EXCL — and only then are the run's tmp directory created and a process started, so a colliding id is retried before anything exists to undo and no existing run's record or log can be overwritten. Any failure after the reservation removes it again (and kills and reaps the process group if one had already started); worktree-validation failures wrap ErrInvalid, so a caller can tell a bad request from a failure of the machine. The one thing no failure removes is the run's tmp directory once it exists — failing to create it is an ordinary spawn failure that leaves nothing behind, but a failure after that point rolls back the registry entry and the log file only.
  • Stop — SIGTERM to exactly one run's recorded process group, bounded wait, then SIGKILL that same group.
  • Poller — periodically recomputes every run's derived state and stamps the exit time of newly-dead runs; it signals nothing.
  • It is also the logging seam for process lifecycle actions (spawned, stopped, exit-observed), written to the stdlib logger the daemon shares; internal/run stays silent.

• internal/logfmt — how any value the binary did not author (run id, project, source, worktree, argv word, journal-derived phase) is rendered into a log line: Go-quoted, so it cannot split the line or forge another, and with the daemon's lifecycle words (started, shutdown) escaped, so it cannot forge one of those either. internal/supervisor and internal/api both log through it, because they log to the same stream.

• internal/api — the HTTP control plane over a registry, and the thin client for it. Both halves live in one package so the request shape, the status codes, and the {"error":"..."} error document stay a single reviewable contract. The seven endpoints and their response shapes are the table under HTTP API; what that table does not say:

  • The read endpoints recompute all five derived fields (alive, phase, gate_pending, last_event_at, schema) at the moment of the call, so a run's state is never served as it was last written to disk.
  • POST /runs checks all four fields itself before calling supervisor.Spawn, and an empty cmd means an unusable argv[0] — [""] is rejected exactly as [] is, while ["sleep",""] (an empty argument) is passed through. A worktree that is not an existing directory arrives as supervisor.ErrInvalid, which is what makes it a 400 rather than a 500.
  • POST /runs/{id}/stop re-derives liveness from /proc first, so supervisor.Stop is not even called for the 409 — nothing is signalled on either error path.
  • GET /runs/{id}/log reads its tail by seeking from the end rather than buffering the file, and clamps an oversized tail rather than refusing it: the parameter says how much of the log the caller wants, not how big the log is allowed to be.
  • GET /runs/{id}/events passes the producer's JSONL lines through verbatim — nothing is parsed and re-serialized — and takes its ordering from internal/events rather than re-deriving one: "the last N events, newest last" is exactly the last N elements of the slice the scanner returns. There is no CLI subcommand for it yet; api.Client.GetEvents is how a caller reaches it.
  • Client — Healthz, ListRuns, GetRun, SpawnRun, StopRun, GetLog, GetEvents; any non-2xx response becomes a *StatusError carrying the status and the daemon's message, whether it arrived as JSON or as plain text.
  • Serve — the daemon lifecycle. It logs what it inherited (one re-attached line per still-alive run, plus a count of the dead ones it holds), then one started line, serves the API until its context is cancelled, and on cancellation logs one shutdown line and then drains in-flight requests, stops the poller and joins it, and saves the registry once — in that order. A poller that fails is logged and restarted rather than taking the daemon down with it, and no supervised run is ever signalled: stopping the daemon does not stop the runs. The listener is injected rather than an address bound here, so the caller owns the bind (loopback by default) and tests can serve on port 0. Translating SIGINT and SIGTERM into that cancellation belongs to the caller, as does pointing the stdlib logger at stdout — Serve only writes to it. Values it did not author (run ids, project names, journal-derived phases, a poller's error text) go through internal/logfmt, as do internal/supervisor's lifecycle lines — the two packages share one log stream, so registry, request or journal content can neither split a log line nor forge a second started or shutdown anywhere in it.

• cmd/millhouse — the binary (make build → build/millhouse). The subcommand is os.Args[1], with no flags before it: each subcommand owns its own flags, so --addr is the listen address for serve and the daemon address for the clients. A missing or unknown subcommand prints usage and exits 2; millhouse help (or -h) prints the same text and exits 0.

  • millhouse serve [--addr ADDR] [--state-dir DIR] [--poll-interval D] — resolves the state dir (--state-dir, else $MILLHOUSE_DIR, else ~/.millhouse), points the stdlib logger at stdout, opens the registry, starts the poller, listens on ADDR (default 127.0.0.1:7777, used exactly as given) and serves the API until SIGINT or SIGTERM, which it turns into the context cancellation internal/api.Serve shuts down on. Stopping the daemon does not stop the runs. There is no authentication in this phase: a non-loopback --addr exposes full control of the runs on this machine to anyone who can reach it.
  • millhouse spawn --project P --source S --worktree WT [--addr ADDR] -- CMD [ARG...] — starts a run through the daemon and prints its id. All three flags are required, and so is the literal --: the command is exactly the words after it (at least one), passed as a list and exec'd directly, never as a shell line. Leaving the -- out is a usage error even when a command follows it, because spawn --project P ... sleep 30 and spawn --project P ... -- sleep 30 are otherwise indistinguishable to the flag package.
  • millhouse list [--addr ADDR] — a fixed-width table of every run the daemon knows about, oldest first: ID, PROJECT, SOURCE, PID, PHASE, GATE, STATE. STATE is running or stopped, derived from the liveness the daemon recomputed for that request and from nothing else. GATE is ! when that run's recomputed gate_pending is true — it is waiting at a gate — and an empty cell when it is not; it says only that a run is waiting, never what for, and nothing in this phase acts on it.
  • millhouse status <id> [--addr ADDR] — that one run's full record as indented JSON, exactly as the API reports it.
  • millhouse stop <id> [--addr ADDR] — stops exactly the one run named, printing stopped <id> or stopped <id> (escalated to SIGKILL). One id, always: there is no way to stop more than one run with one invocation.
  • Every client subcommand takes --addr (default 127.0.0.1:7777) as the daemon's address, and they share one exit-code contract: 0 on success, 1 on any API or network error with the message on stderr, 2 for a usage error — which sends nothing at all. A client's flags may be written anywhere among its arguments, so millhouse status <id> --addr A works as documented: the clients split their raw arguments into flags and positionals themselves rather than letting flag.FlagSet.Parse stop at the first bare word and drop the flags after it. spawn's literal -- is the one boundary that holds: everything after it is the command's argv, flag-shaped or not.
  • This is also the only place that reads $MILLHOUSE_DIR or the home directory, and the only call in the program that sets the stdlib logger's output — everything below cmd/ takes what it needs as a value.

internal/run and internal/supervisor take their state directory as a value, and internal/api is constructed from the very same Registry — no globals, no environment or home-directory reads — so tests run fully isolated in temporary directories (and against httptest, never a fixed port).

prototype/ is the frozen lifecycle probe that validated this model. It is kept compiling (go build ./... covers it) but is historical reference material only: it is byte-frozen, carries no tests, and must not be imitated — its probe scripts deliberately contain the hazardous pattern-matching process ops that the registry exists to replace.

Contributing

AGENTS.md holds the binding repository conventions (stdlib-first, the process-supervision invariants, testing rules), and docs/agents/skills/ holds durable lessons from previous runs. Read both before changing anything here.