Orchestration: unify the coordinator DAG with the Orcastrator director?

[zaridan]

Status: design proposal + a standalone bugfix, prototyped on a fork behind experimentalOrchestrators. Seeking direction before investing in the larger change.

TL;DR

Orca currently has two orchestration systems that share a name but don't compose:

1. Orcastrator director (orchestrator-launch.ts, OrchestratorMissionControl.tsx): a hidden worktree runs a Claude session that plans work and creates child worktrees per PR-track. Mission Control discovers workers purely by worktree lineage and reads .orcastrate/log.jsonl. It never calls the orchestration backend.
2. Coordinator / task DAG (src/main/runtime/orchestration/): a main-process coordinator runs a SQLite task DAG and dispatches tasks to terminals inside a single worktree. It never creates worktrees, and nothing in the Orcastrator UI reads its DB (OrchestratorMissionControl.tsx even comments "no orchestration RPC yet").

We'd like to unify them so the coordinator can drive worktree-based workers that Mission Control can see — which unlocks a live DAG "Control Panel" and an optional token-free "recipe" director. Before we build that, a few questions for maintainers (below), and one bugfix we'd send regardless.

Ship-now bugfix (independent of the proposal)

Multiple coordinators sharing one orchestration DB poach each other's work. The orchestration DB is a single global namespace: tasks, dispatch_contexts, messages, decision_gates have no run scoping, the coordinator handle defaults to the literal 'coordinator', and run-start (getActiveCoordinatorRun → createCoordinatorRun in orchestration-gates.ts) is a TOCTOU race. Two coordinators consume each other's worker_done messages (silent hang) and trip the per-assignee dispatch guard ("already working on the same thing").

Fix: per-run isolation (per-run DB file or coordinator_run_id on all run-scoped tables) + per-run coordinator handle + atomic run-start. We have this in progress on our fork and would send it as its own PR, independent of the proposal below.

The proposal (the part we want direction on)

Make the Orcastrator a director with two backends over the same coordinator/DAG:

• LLM director — today's behavior.
• Recipe director — token-free: compiles a fixed workflow into the DAG and runs it with no LLM in the director worktree. (Justified by determinism/reproducibility for known-shape jobs, not token cost — the director is a small fraction of a run's tokens.)

Both feed one Control Panel that renders the live DAG using Orca's existing visual vocabulary (WorkspaceCard / AgentRow / AgentStateDot).

Required changes, all of which appear additive/targeted:

1. Coordinator creates worktrees. Add createWorktree to CoordinatorRuntime, implemented on OrcaRuntimeService by wrapping the existing addWorktree (src/main/git/worktree.ts). Workers run in their own worktrees with lineage parent = director, so Mission Control sees them.
2. Mission Control reads the DAG. Consume listTasksWithDispatch (already exists in db.ts) instead of pure lineage.
3. Coordinator resume-on-boot. A startup hook that resumes or reconciles coordinator_runs WHERE status='running' — today the loop is in-memory with no resume (fine while a human-driven agent re-issues run, but not for a headless director).
4. Renderer→main orchestration.run binding (today CLI-only).
5. Per-run isolation (the bugfix above) — prerequisite.

Questions for maintainers

1. Is the terminal-dispatch-in-one-worktree model of the coordinator intentional, or an artifact? Any reason not to have it create worktrees per track?
2. Do you want these two systems unified, or kept deliberately separate (LLM director = worktree/lineage; coordinator = terminal DAG for a different use case)? If separate, we'd scope the Control Panel narrowly.
3. Appetite for a token-free "recipe" director at all, or is the LLM director the only intended planner?
4. Anything in flight that would collide with per-run DB isolation or a CoordinatorRuntime.createWorktree?

Happy to build it on a fork and bring a working demo — we'd just rather align on (1)–(3) before investing in the bridge.

[zaridan]

Quick update on the ship-now bugfix from the original post — it's now implemented and merged on our fork (2c2058f48).

What it does, independent of the larger proposal:

• Per-target run-scoping of the orchestration DB: coordinator_run_id on tasks/dispatches/gates, plus target_key on coordinator_runs and tasks so two coordinators on different repos/worktrees run in parallel while a duplicate run on the same target is refused. adoptUnownedTasks and taskCreate are target-scoped, closing cross-target task poaching.
• Per-run coordinator handle (instead of the literal 'coordinator') so worker→coordinator mail can't be consumed by another run.
• Atomic run-start via BEGIN IMMEDIATE (cross-process, since the old in-memory guard didn't span runtimes — which is how the clash happened). Target-key resolution fails closed rather than silently splitting.
• Tests include a real two-connection-on-one-file race and an adoption-under-concurrency case.

It stands entirely on its own — happy to open it as a standalone PR here whenever you'd like, separate from the bridge proposal.

Still very interested in your read on questions 1–3 above (especially: is the terminal-dispatch-in-one-worktree model intentional, and do you want the two systems unified?) before we invest in the larger change. We're prototyping the bridge on the fork and can bring a working demo.

[zaridan]

Update — this is now built and up for review as #6293.

Rather than wait on direction for the bridge, we built the unified version on our fork and have opened it as a single, cohesive PR: #6293. It implements items 1–5 from the proposal (coordinator-created worktrees with director lineage, Mission Control reading the live DAG, coordinator resume-on-boot, the renderer→main orchestration.run binding, and per-run DB isolation — the ship-now bugfix, already merged on our fork per the comment above).

One deliberate divergence from the proposal: we dropped the token-free recipe director (Question 3). In practice the director is a small fraction of a run's tokens, so the determinism argument didn't justify a second backend — the shipped version is recipe-free, LLM-director only. The only residual recipe field is a mode-agnostic, always-null snapshot column.

Happy to take direction on the open questions directly on the PR. Closing this out as resolved.