Swarm Overlord

Swarm Overlord

The Overlord is the cross-project meta-orchestrator of Nebulus Swarm. Beyond managing individual minions, it understands project relationships, dependency graphs, and blast radius across your entire ecosystem.

Architecture

The Overlord operates in three modes:

1. Meta-Orchestrator (Phases 1-2, 279 tests): Project discovery, dependency tracking, scope analysis, and autonomy
2. Slack + Background Mode (Phase 3, 285 tests): Slack command router, approval workflows, background daemon, proactive detection, notifications
3. Control Plane (Original): Slack/Docker minion dispatch and coordination

590 total Overlord tests across all phases (v2.6.0).

Phase 1-2: Meta-Orchestration

Registry

Project configuration and dependency management using overlord.yml files:

• Discovers all projects in your workspace
• Loads project metadata (name, path, dependencies, test commands)
• Builds dependency order using Kahn's algorithm
• Validates circular dependencies

Scanner

Git state and health monitoring across all projects:

• Scans for uncommitted changes, unpushed commits
• Detects active branches and their status
• Runs test suites and reports health
• Tracks last commit timestamps

Dependency Graph

DAG traversal and impact analysis:

• Find upstream dependencies (what this project depends on)
• Find downstream dependents (what depends on this project)
• Calculate affected projects by a change
• Render ASCII dependency trees
• Extract subgraphs for specific projects

Action Scope

Blast radius evaluation for changes:

• Categorizes changes: local, upstream, downstream, cross-cutting
• Computes affected project sets
• Evaluates autonomy suitability (can this be auto-dispatched?)
• Confidence scoring based on scope and test health

Memory

Cross-project observation store (SQLite):

• Remember key decisions, patterns, gotchas across projects
• Search observations by keyword or project
• Tag observations with categories and confidence
• Prune stale observations
• Export/import for sharing

Shared via nebulus-core: The canonical OverlordMemory implementation now lives in nebulus_core.memory.overlord (pure stdlib, no extra dependencies). Atom's nebulus_swarm/overlord/memory.py is an import shim that tries nebulus_core first and falls back to a local copy for standalone installs. All consumer code continues importing from nebulus_swarm.overlord.memory unchanged.

Autonomy Engine

Confidence-based auto-dispatch logic:

• Scores changes on: scope, test health, recent failures, observation history
• Recommends human review vs. auto-dispatch
• Suggests reviewer assignments based on affected projects
• Tracks dispatch outcomes for learning

Dispatcher (GateFlow Pattern)

The Dispatcher manages the execution lifecycle of tasks using the GateFlow (Verify-Fix-Re-verify) pattern. It ensures high reliability by triaging worker output and automatically retrying failed reviews.

• Structured Return Protocol: Workers emit a machine-parseable ---NEBULUS-RETURN--- block at the end of their output.
  • Fields: STATUS, SUMMARY, FILES_CREATED, FILES_MODIFIED, TESTS_PASSED, TESTS_FAILED, BLOCKERS, NEXT_ACTION.
  • Triage: The Dispatcher parses this block to detect blocked or error states immediately, failing the task with diagnostic context before proceeding to the formal review step.
• Verify-Fix Loop: If a task passes triage but fails the subsequent review step, the Dispatcher enters a fix pass.
  • A FixContext is built containing the original brief, review feedback, attempt count, and previous output.
  • A new MISSION_BRIEF is generated with this context to guide the worker's fix attempt.
• Retry Policy:
  • Max Retries: Defaults to 2 (configurable per-task via work_queue.tasks.max_retries).
  • Scope: Only review failures trigger retries; execution failures (e.g., API errors, timeouts) fail immediately to prevent infinite loops and budget waste.

Phase 3: Slack + Background Mode

Slack Command Router (112 tests)

Multi-project Slack command routing (slack_commands.py):

• status [project] — Ecosystem or project health via scanner
• scan [project] — Detailed scan with issue detection
• merge <project> <source> to <target> — Dispatch via TaskParser with approval
• release <project> <version> — Coordinated release workflow
• autonomy [level] — Show/change autonomy level
• memory <query> — Search cross-project memory
• help — List available commands

Proposal Manager (31 tests)

Approval lifecycle with Slack thread binding (proposal_manager.py):

• States: pending → approved → executing → completed/failed (or denied/expired)
• Posts proposals to Slack with scope summary
• Reply "approve" or "deny" in thread to control execution
• Auto-expires pending proposals after configurable TTL
• SQLite-backed persistent proposal store

Background Daemon (37 tests)

Persistent daemon process with scheduled sweeps and PID-based lifecycle management (overlord_daemon.py):

• PID file at ~/.atom/overlord/daemon.pid for reliable process discovery
• croniter-based scheduler for configurable task execution
• Scheduled tasks: scan (hourly), test-all (nightly), clean-stale-branches (weekly)
• Graceful shutdown via SIGINT/SIGTERM with PID file cleanup
• Automatic Slack bot connection when tokens are available
• CLI: nebulus-atom overlord daemon start|status|stop|restart

Proactive Detectors (24 tests)

Issue detection patterns (detectors.py):

• StaleBranchDetector — Branches with no activity for N days
• AheadOfMainDetector — Develop branches with unmerged commits
• FailingTestDetector — Projects with test failures or missing tests
• Detection engine with autonomy-level filtering
• Integrated into scheduled scans and on-demand @atom scan

Notification System (19 tests)

Alert routing and daily digest (notifications.py):

• Urgent notifications — Immediate Slack posts for critical events
• Buffered accumulation — Category-based buffering for daily digest
• Daily digest — Formatted summary with health checks, detections, proposals, executions
• Configurable via NotificationConfig in overlord.yml
• Category tracking: detections, proposals, executions, health checks, test sweeps

Phase 0: Control Plane Components

Overlord Control Plane
├── SlackBot          # Slack integration (Socket Mode)
├── LLMCommandParser  # Natural language command parsing
├── DockerManager     # Container lifecycle management
├── GitHubQueue       # Issue scanning and label management
├── OverlordState     # SQLite state persistence
├── ModelRouter       # Complexity-based model selection
└── ReviewWorkflow    # Automated PR review (optional)

Slack Integration

The Overlord connects to Slack using Socket Mode (no public URL required). It listens for messages in a configured channel and routes them through the command parser.

Message Flow

Slack message → SlackBot → LLMCommandParser → Command handler → Response

The LLM parser uses a small model (default: llama-3.1-8b) to interpret natural language commands with a 5-second timeout. Falls back to regex parsing if the LLM is unavailable.

Thread Replies

The Overlord tracks Slack threads for the clarifying questions feature. When a minion asks a question, the Overlord posts it as a Slack message. Human replies in that thread are captured and forwarded back to the minion.

Docker Management

The DockerManager handles the full container lifecycle:

• Spawn: Creates containers with environment variables for the minion's task
• Monitor: Tracks active containers and their status
• Kill: Terminates containers on request or timeout
• Cleanup: Removes dead/exited containers
• Sync: Recovers container tracking after Overlord restart

Each minion container runs in the nebulus-swarm Docker network with resource limits (2GB RAM, 1 CPU).

State Management

SQLite database (/var/lib/overlord/state.db) tracks:

• Active minions: ID, repo, issue, status, heartbeat timestamps
• Work history: Completed/failed tasks with duration and error details
• Minion lookups: By ID, by issue, active list

Key Methods

Method	Description
add_minion()	Register a new minion
update_minion_status()	Update status and heartbeat
get_active_minions()	List all active minions
get_minion_by_issue()	Find minion working on an issue
get_work_history()	Query history with filters
get_distinct_repos()	List unique repositories

Queue Management

The GitHubQueue scans watched repositories for issues labeled nebulus-ready:

1. Fetch all open issues with the work label
2. Skip issues already in-progress
3. Sort by priority (high-priority label) then creation date
4. Return as QueuedIssue objects

Cron Sweeps

When cron is enabled (default: daily at 2:00 AM), the Overlord:

1. Scans all watched repos for ready issues
2. Checks available minion slots
3. Warms up the LLM server
4. Routes each issue through the Model Router
5. Spawns minions for top-priority issues
6. Notifies Slack of each spawn

HTTP Endpoints

The Overlord exposes health and API endpoints on port 8080:

Endpoint	Method	Description
/health	GET	Health check (returns 200)
/status	GET	System status, active minions, pending questions
/queue	GET	Cached queue scan results
/minion/report	POST	Minion event callbacks
/minion/answer/{id}	POST	Answer a minion's question

/status Response

{
  "paused": false,
  "active_minions": [...],
  "docker_available": true,
  "config": {
    "max_concurrent": 3,
    "timeout_minutes": 30
  },
  "pending_questions": [...]
}

Clarifying Questions

When a minion encounters ambiguity, the Overlord:

1. Receives a QUESTION event from the minion
2. Posts the question to Slack
3. Stores it as a PendingQuestion with the thread timestamp
4. Waits for a human to reply in the Slack thread
5. Routes the reply back to the minion via the answer endpoint

Constraints: max 3 questions per minion, 10-minute timeout per question.

Watchdog

A background task monitors minion health:

• Checks heartbeat timestamps every 60 seconds
• Kills minions that haven't sent a heartbeat within the configured timeout (default: 5 minutes)
• Notifies Slack of timed-out minions
• Cleans up dead Docker containers

Configuration

See Configuration for all Overlord environment variables.

Related Pages

• Nebulus Swarm - System overview
• Swarm Minion - Worker agent details
• Model Router - How models are selected
• Swarm Dashboard - Monitoring the Overlord