feat: conductor foundations — session resolution (P0–P1) + durable owner-delegated identity (P2)

docs/conductor-prior-art-firstmate.md

@@ -0,0 +1,153 @@
+# Prior Art: firstmate — what to borrow, where codeoid is better
+
+> [firstmate](https://github.com/kunchenguid/firstmate) (by @kunchenguid) is the
+> single most relevant prior art for the conductor: a **shipped** implementation
+> of the exact "talk to one agent, it runs a crew" model, dogfooding the author's
+> own tools (treehouse worktrees, herdr, Orca). It is the **architectural inverse**
+> of codeoid — which is precisely why it's worth studying. Analysis based on its
+> `docs/architecture.md`, `AGENTS.md` §1 (identity) + §8 (supervision), the 40+
+> `bin/` scripts, and its skills.
+
+## The mirror
+
+Both systems are the same idea: one liaison you talk to, a crew of autonomous
+workers, worktree isolation, approval-gated merges, restart-proof state. The
+substrate is opposite:
+
+| Axis | firstmate | codeoid conductor |
+|---|---|---|
+| Orchestrator | **`AGENTS.md` (122 KB prompt) + bash helpers** — "a directory that turns any agent into your firstmate" | **TS/Bun daemon** — logic in code, thin prompts |
+| Worker isolation | git worktree (treehouse) per task, visible tmux/herdr/zellij/Orca window | daemon-owned session per task |
+| State owner | disk (`data/`, `state/`) + the session backend | the daemon (clients are pure renderers) |
+| Session/task resolution | **human names it; conductor greps `data/backlog.md`** — no semantic search | **hybrid retrieval + rerank over cards** (the P1 linchpin) |
+| Identity/safety | "guarded by construction" + git auth + merge approval — **no crypto identity** | **ZeroID/WIMSE per session + delegation chain + (later) Cedar/Shield** |
+| Supervision | zero-token bash watcher; absorbs benign wakes, wakes LLM only on actionable events | (to build — see P4) |
+| Harness | **any** (claude/codex/opencode/pi/grok) via adapters | Claude-first, going meta-harness |
+
+The headline: firstmate **needs** a 122 KB prompt precisely *because* it has no
+daemon to enforce invariants — every rule ("never write to a project", "keep one
+live watcher cycle") is a fragile instruction the model must remember every turn.
+Codeoid enforces those in code. So firstmate is both a **feature catalog to mine**
+and a **proof of what you pay if you skip the daemon.**
+
+## STEAL — ranked by leverage
+
+### 1. The conductor is read-only over targets *by construction*
+firstmate Hard Rule #1: "**Never write to a project.** You read projects to
+understand them; crewmates change them." Only 6 narrow, all-fast-forward/guarded
+write exceptions exist. This is a **stronger invariant than our R3** ("confirm
+before send-class acts") because it's architectural, not a prompt discipline.
+→ **Adopt:** the conductor's `codeoid_fleet` tool surface is **read + dispatch
+only** — no file/git/shell-write tool on target repos ever. All mutation flows
+through spawned crewmates behind approval. The blast-radius bound becomes a
+property of the tool surface, and codeoid can *enforce* it (deny those tools to
+the conductor identity via scopes), where firstmate can only *ask* for it.
+
+### 2. Zero-token, event-driven supervision (their crown jewel)
+A cheap watcher (`fm-watch.sh` + `fm-classify-lib.sh` + `fm-crew-state.sh`)
+classifies every wake in bash, **absorbs the benign majority** (`working:` notes,
+no-change heartbeats, provably-working stale panes) without ever spending an LLM
+turn, and wakes the conductor **only on actionable events** (`needs-decision` /
+`blocked` / `failed` / `done` / `PR ready` / `merged`). Idle fleet = zero tokens.
+Heartbeats **back off exponentially** (600 s → 2 h cap). Actionable wakes hit a
+**durable queue** before detector state advances, so a missed exit is recoverable.
+→ **Adopt (and do it better):** this is the answer to our "never OOC + cost" goal
+and our P4 "event-driven digests." Codeoid's daemon already owns session state, so
+it can **push** real events — no bash polling, no pane-tail regex. Conductor LLM
+turns fire only on actionable daemon events; a "provably working" predicate gates
+absorption; heartbeat backstop with exponential backoff; durable event queue for
+crash recovery.
+
+### 3. Two task shapes: **ship** vs **scout**
+ship = deliver a change (PR / local-merge → teardown); scout = investigate / plan /
+reproduce / audit → report at `data/<id>/report.md`, never pushes, worktree is
+scratch from the start.
+→ **Adopt:** dispatch carries a `shape`. Scout results are reports (great for "go
+find out X" without touching code); ship results are PRs/merges; teardown rules
+differ. Clean taxonomy our P4 lacked.
+
+### 4. Per-project autonomy modes (not blanket confirm)
+`data/projects.md` gives each project a mode — `no-mistakes` / `direct-PR` /
+`local-only` — plus optional **`+yolo`** ("make routine approval decisions
+yourself; destructive/irreversible/security-sensitive still escalates").
+→ **Adopt:** replace our blanket R4 "owner approval only" with **per-workspace
+autonomy policy**. This maps *perfectly* onto codeoid's identity/Cedar future — a
+project's mode literally *is* a policy bound to the conductor's identity. Low-risk
+repos run hands-off; sensitive repos require approval.
+
+### 5. `/afk` batched-digest away-mode + `/stow` knowledge sweep
+`/afk` hands supervision to a daemon that self-handles routine wakes and escalates
+**only captain-relevant events as one batched, single-line digest** — cutting cost
+while you step away. `/stow` sweeps the session for durable knowledge and routes
+each finding to its disk home (prefs → `captain.md`, gotchas → `learnings.md`,
+project knowledge → project `AGENTS.md`, task notes → backlog).
+→ **Adopt** both as conductor UX: an away-mode that batches escalations, and a
+handoff/knowledge-capture that routes findings to durable homes (codeoid already
+has the memory engine to route into).
+
+### 6. Harness dispatch profiles
+`config/crew-dispatch.json` — natural-language rules the conductor reads at intake
+to pick `--harness/--model/--effort` per task; the shell validates the shape, the
+LLM matches intent.
+→ **Adopt:** directly feeds codeoid's meta-harness direction — the conductor routes
+each task to the best backend (claude/codex/gemini) per NL rules + a validated
+config.
+
+### 7. Sentinel marker for system-injected messages
+Daemon escalations injected into the conductor's chat are prefixed with
+`FM_INJECT_MARK` (ASCII unit-separator `0x1f`) so the conductor can tell an
+internal escalation from a real captain message.
+→ **Adopt:** our conductor faces the same ambiguity (daemon event-digests vs. real
+messages from web/Telegram/mobile). Use an out-of-band field or sentinel so
+injected events are never confused with user input.
+
+### 8. Secondmates = the nested-conductor scaling path (future)
+Persistent **domain supervisors** that are "ordinary direct reports run from
+isolated homes" — "there is no second architecture; a secondmate is a crewmate
+whose workspace is an isolated home and whose brief is a charter."
+→ **Note:** validates that our single-global-conductor choice can grow *domain
+sub-conductors* later using the **same delegation-depth identity chain** we already
+designed (human → conductor → sub-conductor → crewmate). No new architecture.
+
+Also worth borrowing: **restart-proof reconcile + durable wake-queue**, and the
+**status-vs-current-state discipline** (`fm-crew-state.sh` reconciles an
+authoritative run-step over a possibly-stale status line — a worker that reported
+`done:` before a long validation isn't actually done).
+
+## Where codeoid is already better — keep these
+
+1. **Identity-native.** firstmate has *no* cryptographic identity; safety is prompt
+   discipline + git auth + merge approval. For the "master of my machine" ambition
+   (email, arbitrary tools, many agents), codeoid's per-session ZeroID + delegation
+   + (later) Cedar/Shield is a real moat firstmate can't match. Its worker
+   isolation is filesystem homes; ours is cryptographic delegation with cascading
+   revocation.
+2. **Semantic session resolution.** firstmate resolves "which session" via the
+   human naming it + the conductor grepping a markdown backlog. It has **no**
+   embedding/rerank/hybrid recall — exactly the P1 linchpin we're building. This is
+   a genuine advance over the most mature conductor in the wild.
+3. **Determinism — logic in code, not a 122 KB prompt.** Orchestration invariants
+   live in testable TS, not re-read-every-turn prose. Cheaper context, deterministic
+   behavior, unit-testable (see our P0 tests).
+4. **Daemon-native events beat bash polling.** firstmate scrapes tmux panes with
+   regex because tmux "has no native primitive and always reports unknown." Our
+   daemon owns lifecycle → real push events, no scraping.
+
+## Anti-patterns to avoid
+
+1. **Prompt-as-program.** Don't drift toward encoding conductor logic in a giant
+   `AGENTS.md`. Keep it in code; prompts stay thin.
+2. **Pane-tail-regex liveness.** Never guess worker health by scraping terminal
+   output — use daemon-authoritative state.
+3. **Status side-channels that go stale.** firstmate spends pages reconciling
+   "status log says done but a run is active." Don't reintroduce a lossy status
+   stream parallel to the daemon's authoritative state.
+
+## Net
+
+The comparison **validates codeoid's architecture** (daemon + identity + structured
+memory) as the right foundation, and firstmate proves the conductor concept ships.
+The highest-leverage borrows are **behavioral/design, not code** (opposite
+substrate): read-only-by-construction (#1), zero-token event supervision (#2),
+ship/scout shapes (#3), per-project modes (#4). Fold #1–#4 into the design now;
+#5–#8 are UX/scaling adds for P4–P7.

docs/conductor-prior-art-hermes.md

@@ -0,0 +1,153 @@
+# Prior Art: hermes-agent — what to borrow, where codeoid is better
+
+> [hermes-agent](https://github.com/NousResearch/hermes-agent) (Nous Research,
+> MIT) is the most *complete* personal-assistant prior art we've studied — a
+> multi-platform gateway (Telegram/Discord/Slack/WhatsApp/Signal/email), cron
+> **routines**, autonomous skill creation, cross-session memory + user modeling,
+> `delegate_task` + a durable **Kanban** work-queue, all running on a $5 VPS. It is
+> the closest thing to the original "master of my machine" ask. Analysis from its
+> `README`, `docs/session-lifecycle.md`, `hermes-already-has-routines.md`,
+> `AGENTS.md` (Delegation / Curator / Cron / Kanban), and the subsystem layout.
+
+## Triangulation — where codeoid sits
+
+Three prior-arts, three niches:
+
+| System | Niche | Security model | Substrate |
+|---|---|---|---|
+| **OpenClaw** | channel-gateway breadth (front door + life-admin) | allowlist | files + bash |
+| **hermes** | **most complete personal assistant** (gateway + routines + skills + memory + user model) | allowlist + DM-pairing + command-approval | Python monolith (`cli.py` 738 KB) |
+| **firstmate** | *coding-fleet* conductor (crew + worktrees + PRs) | guarded-by-construction + merge approval | 122 KB prompt + bash |
+| **codeoid** | **identity-native + retrieval-first + typed multi-client daemon** | **ZeroID/WIMSE per-session + delegation** | TS daemon, clients-are-renderers |
+
+hermes and firstmate each hold one or two legs; **codeoid is the only one with all
+three** (crypto identity + semantic session resolution + typed multi-client daemon).
+That triad is the defensible position. hermes serves the *personal-assistant
+breadth* better than firstmate — which is exactly why it's worth mining for the
+"master of my machine" surface our plan was thin on.
+
+## STEAL — ranked by leverage
+
+### 1. Routines — scheduled + triggered autonomy (the biggest gap in our plan)
+hermes has cron **and** webhook/event triggers, in natural language:
+`hermes cron create "0 2 * * *" "triage the backlog" --deliver telegram`;
+`hermes webhook subscribe pr-review --events pull_request --prompt "…"` (HMAC-auth).
+Per-job fields: `skills`, `model`/`provider` override, `script` (pre-run
+data-collection whose stdout is injected — or `no_agent=True` to make the script
+the *whole* job), `context_from` (chain job A's output into job B), `workdir` (run
+in a repo with its `AGENTS.md` loaded), multi-platform delivery.
+`cron/jobs.py` (store) + `cron/scheduler.py` (tick loop).
+**Two hardening patterns to steal whole:**
+- **Script-injection + `[SILENT]` pattern** — a script does the mechanical work
+  (fetch/diff/compute), the agent only *reasons*, and the job emits nothing unless
+  something changed (`respond with [SILENT]`). Zero-spam, near-zero-token monitors.
+- **Cron hardening** — 3-minute hard interrupt (runaway loops can't monopolize the
+  scheduler), file-lock (`.tick.lock`) against duplicate ticks across processes,
+  catchup/grace windows, `skip_memory=True` by default on cron sessions, and cron
+  output lands in its *own* session (header/footer frame) so it never corrupts the
+  main conversation's role alternation.
+→ **New conductor phase (P4.5).** Neither codeoid nor firstmate has this, and it's
+core to "master of my machine" (nightly triage, monitors, digests, event triggers).
+
+### 2. Durable Kanban work-queue as the dispatch backbone
+`AGENTS.md` §Kanban + `tools/kanban_tools.py`: a **SQLite-backed board** with a
+**dispatcher loop** (default 60 s) that reclaims stale claims, promotes ready tasks,
+**atomically claims**, and spawns the assigned worker. **Board = hard boundary**
+(workers get `HERMES_KANBAN_BOARD` pinned in env, can't see other boards); tenant =
+soft namespace within a board. After `failure_limit` consecutive failures (default
+2) the dispatcher **auto-blocks the task** to prevent spin loops.
+→ Far more restart-proof than firstmate's markdown backlog. **Upgrade P4's dispatch
+to sit on a durable queue like this** (codeoid already has SQLite): atomic claim,
+stale-claim reclaim, failure-limit auto-block (complements firstmate's stuck-loop).
+
+### 3. `delegate_task` role model — leaf vs orchestrator
+`tools/delegate_tool.py`: a subagent gets an isolated context + terminal.
+`role="leaf"` (default) is a focused worker that **cannot** call `delegate_task`,
+`memory`, `send_message`, `execute_code`; `role="orchestrator"` **can** spawn,
+bounded by `max_spawn_depth` (default 2) and `max_concurrent_children` (default 3).
+Background delegation returns an id immediately and re-enters via an async
+completion queue. Durability rule: background delegate is process-local — for
+restart-survival use a cronjob or `terminal(background, notify_on_complete)`.
+→ This *is* our read-only-by-construction + delegation-depth decisions, with the
+concrete knobs — except **codeoid enforces the role's capability restriction
+cryptographically via ZeroID scopes**, where hermes uses config flags. Adopt the
+leaf/orchestrator split; enforce it at the scope layer, not by prompt or flag.
+
+### 4. Session-lifecycle hardening (`docs/session-lifecycle.md`)
+A mature, battle-tested state machine worth mining for reliability:
+- **Restart recovery:** `resume_pending` (soft — preserve `session_id`, continue the
+  transcript) vs `suspended` (hard wipe); `suspend_recently_active(120s)` on a crash
+  (no `.clean_shutdown` marker); a `.clean_shutdown` marker skips resurrection after
+  a clean restart.
+- **Stuck-loop escalation:** a restart-count file auto-suspends a session active
+  across 3+ consecutive restarts (terminal escalation, complements Kanban's
+  failure-limit).
+- **Agent LRU cache** (128 entries, 1 h idle TTL) that **preserves prompt-cache**
+  across turns; background expiry watcher (5 min) finalizes + evicts.
+- **Burst-collapse message queue:** single "next-up" slot per session (repeat sends
+  overwrite) + FIFO overflow for explicit `/queue`, so multi-message bursts during a
+  turn never process out of order.
+- **Per-session token/cost tracking** baked into the session record.
+→ codeoid's daemon has some of this; the `resume_pending`/stuck-loop/clean-shutdown
+state machine + burst-collapse queue are concrete P4 hardening.
+
+### 5. Multi-platform gateway
+One gateway process → Telegram/Discord/Slack/WhatsApp/Signal/email, with a clean
+`SessionSource` (message-origin descriptor) → deterministic session-key
+(`agent:main:{platform}:{chat_type}:{chat_id}:{thread}:{participant}`) → home
+channels + delivery routing + multi-user isolation + PII-redaction-in-prompt.
+→ How our P5 adds platforms cheaply beyond web+Telegram. Copy the SessionSource +
+session-key shape; codeoid's daemon already owns the session store this plugs into.
+
+### 6. Zero-context-cost tool-RPC scripts
+"Write a Python script that calls tools via RPC, collapsing multi-step pipelines
+into one zero-context turn." Complements codeoid's saar/extraction work and
+firstmate's zero-token supervision — a script surface for mechanical multi-step
+work that never floods the conductor's context.
+
+### 7. Self-improving loop (Curator) — aspirational, clean invariants
+`agent/curator.py`: autonomous skill creation after complex tasks + a maintenance
+loop that tracks per-skill usage and **archives (never deletes)** stale skills,
+exempts pinned, and **only touches `created_by: agent` skills** (bundled/hub skills
+off-limits). Plus Honcho dialectic user modeling ("who you are across sessions").
+→ The direction codeoid's memory engine could grow (autonomous skills + a user
+model); the safe-autonomy invariants (archive-not-delete, pinned-exempt,
+provenance-scoped) are worth copying if/when we add agent-authored skills.
+
+### 8. Notes
+- **ACP** (`acp_adapter`, `acp_registry`, `agent/copilot_acp_client.py`) — hermes
+  speaks the Agent-Client-Protocol. An interop/meta-harness play: codeoid speaking
+  ACP would let editors (Zed/Copilot) drive it.
+- **Serverless-persistence backends** (Modal/Daytona) — hibernate-when-idle so a
+  cloud conductor costs ~nothing between sessions. The cheap-VPS story.
+
+## Where codeoid is already better — keep
+
+1. **Identity.** hermes is allowlist + DM-pairing + command-approval — no
+   cryptographic per-session identity or delegation chain. For "master of my
+   machine" with email + arbitrary tools + a fleet, codeoid's ZeroID is the moat;
+   and it lets us enforce the delegate role model (#3) *cryptographically* rather
+   than by config flag.
+2. **Retrieval.** hermes cross-session recall = FTS5 + LLM summarization (better
+   than firstmate's markdown grep, but no rerank or bi-temporal cards). Our P1
+   (BGE-M3 hybrid + cross-encoder rerank + bi-temporal cards) is more sophisticated.
+3. **Typed daemon + modularity.** hermes is a Python monolith with 250–738 KB
+   god-files. codeoid's typed TS daemon + clients-are-renderers keeps the
+   multi-client (web/TUI/mobile) story crisp and the code testable.
+
+## Anti-patterns to avoid
+1. **God-files** (`cli.py` 738 KB, `run_agent.py` 268 KB, `hermes_state.py` 255 KB)
+   — keep the daemon in small typed modules.
+2. **Allowlist-only security** — don't regress from ZeroID to DM-pairing/allowlists.
+3. **Prompt-heavy `AGENTS.md`** (71 KB) — same trap as firstmate, less extreme; keep
+   orchestration in code.
+
+## Net — refinements to the plan
+- **Upgrade P4** — durable Kanban-style work-queue (atomic claim, stale reclaim,
+  failure-limit auto-block) + the `leaf`/`orchestrator` role model (enforced via
+  scopes) + session-lifecycle hardening (resume_pending / stuck-loop / clean-shutdown
+  / burst-collapse queue).
+- **Add P4.5 — Routines** — scheduled + webhook/event-triggered autonomy, with the
+  script-injection `[SILENT]` pattern and cron hardening (hard interrupt, tick lock,
+  own-session output). This is what turns the conductor from a fleet *supervisor*
+  into a personal *assistant*.