Build full worker/orchestrator setup

[harshitsinghbhandari]

Summary

Track the remaining work to rebuild the full worker/orchestrator setup from the original Composio Agent Orchestrator in the Go rewrite.

The immediate MVP is worker/orchestrator communication and role-specific prompts. The full setup still needs canonical orchestrator identity, lifecycle orchestration, dashboard affordances, and richer worker management semantics.

Current state

Already present in the Go rewrite:

• Session kinds: worker and orchestrator
• POST /api/v1/sessions for worker-like session spawn
• POST /api/v1/orchestrators for orchestrator spawn
• ao spawn
• ao send
• zellij-backed live session runtime
• per-session env: AO_SESSION_ID, AO_PROJECT_ID, AO_ISSUE_ID, AO_DATA_DIR
• basic session lifecycle and reaper
• project registry backed by sqlite

MVP slices

• Worker prompt includes a "Talking to the Orchestrator" section when an active project orchestrator exists. Implemented in fix: prefix ao send messages with sender session #85.
• ao send auto-prefixes messages with [from $AO_SESSION_ID] when invoked from inside an AO-managed session. Implemented in fix: prefix ao send messages with sender session #85.
• Orchestrator sessions receive an orchestrator-specific prompt explaining their coordinator role and available commands. Implemented in fix: prefix ao send messages with sender session #85.

Completed MVP work

Implemented in #85:

• ao send prefixes messages with [from <session-id>] when called from inside an AO session.
• Worker spawn prompts include orchestrator contact instructions when a non-terminated same-project orchestrator exists.
• Orchestrator spawn prompts include coordinator-role instructions, including ao spawn and ao send usage.

Full orchestrator setup still left

Canonical orchestrator identity

Original repo uses one deterministic orchestrator per project, usually:

<project-prefix>-orchestrator

Needed work:

• Add a way to reserve/create fixed session IDs instead of only {project}-{num}.
• Make POST /api/v1/orchestrators reuse an existing active canonical orchestrator.
• Add clean/relaunch semantics that terminate the old canonical orchestrator and create a fresh one.
• Ensure legacy numbered orchestrators do not become the canonical one accidentally.

Orchestrator prompt and behavior

• Port/adapt the original orchestrator prompt template.
• Include project context, registered repo, default branch, dashboard/API hints, and command examples.
• Teach orchestrator to spawn workers, send follow-up messages, triage blockers, and wait for worker reports.
• Keep orchestrator focused on coordination unless explicitly asked to implement directly.

Worker prompt and backchannel

• Include exact ao send --session <orchestrator-id> --message "..." command when an orchestrator exists.
• Keep worker-to-orchestrator sends rare and scoped to true blockers/cross-session coordination.
• Ensure sender identity is unambiguous via automatic [from <session>] prefix.
• Consider exposing orchestrator ID in session metadata or read model.

Session manager semantics

• Ensure orchestrator spawn uses branch orchestrator/<session-id>.
• Ensure worker default branch remains unique and worker-oriented.
• Decide whether orchestrator workspaces are reused/adopted or always recreated.
• Prevent orchestrator sessions from claiming PRs or being treated like implementation workers.
• Make cleanup behavior distinguish orchestrator vs worker worktrees.

API / CLI

• Add/adjust CLI command for ao orchestrator start or similar if needed.
• Make ao start optionally ensure project orchestrator.
• Make ao spawn output mention the project orchestrator if present.
• Support clean relaunch from CLI/API.
• Add API tests for reuse, clean relaunch, and canonical conflict behavior.

Dashboard / UX

• Show orchestrator separately from workers.
• Make project page expose launch/relaunch orchestrator action.
• Make worker pages show "message orchestrator" affordance.
• Make orchestrator pages show active workers and their lifecycle states.

Lifecycle / automation

• Route worker reports and PR events to the orchestrator where appropriate.
• Decide which state transitions are SCM-owned vs agent-reported.
• Add review/CI failure forwarding semantics from orchestrator to workers.
• Add safeguards around noisy automatic messaging.

Suggested implementation order

1. MVP communication primitives: worker prompt backchannel, ao send sender prefix, orchestrator prompt.
2. Canonical orchestrator ID/reuse semantics.
3. Clean relaunch semantics.
4. Dashboard and lifecycle routing polish.