Skip to content

Deployment

Jump to bottom
Jason L. West edited this page Feb 10, 2026 · 5 revisions

Deployment

Docker Deployment

Core Agent

Build and run the core Nebulus Atom agent:

# Build the image
docker build -t nebulus-atom:latest .

# Run with Docker Compose
docker compose up -d agent

# View logs
docker compose logs -f agent

Flight Recorder Dashboard

docker compose up -d dashboard

Access at http://localhost:8501.

Full Core Stack

docker compose up -d

Starts both the agent and dashboard services.

Swarm Deployment

Build Images

# Overlord image
docker build -t nebulus-overlord:latest -f nebulus_swarm/overlord/Dockerfile .

# Minion image
docker build -t nebulus-minion:latest -f nebulus_swarm/minion/Dockerfile .

Start the Overlord

# Using Docker Compose
docker compose -f docker-compose.swarm.yml up -d overlord

# Or directly
docker run -d \
  --name overlord \
  -p 8080:8080 \
  -v overlord-state:/var/lib/overlord \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --env-file .env.swarm \
  nebulus-overlord:latest

The Overlord needs access to the Docker socket to spawn minion containers.

Verify Health

curl http://localhost:8080/health
# {"status": "ok"}

curl http://localhost:8080/status
# {"paused": false, "active_minions": [], ...}

Swarm Dashboard

OVERLORD_URL=http://localhost:8080 \
STATE_DB_PATH=/path/to/state.db \
streamlit run nebulus_swarm/dashboard/app.py

Network Architecture

┌─────────────────────────────────────────┐
│            nebulus-swarm network         │
│                                         │
│  ┌───────────┐    ┌──────────────────┐  │
│  │  Overlord │    │   Minion(s)      │  │
│  │  :8080    │◄──►│   (ephemeral)    │  │
│  └─────┬─────┘    └────────┬─────────┘  │
│        │                    │            │
└────────┼────────────────────┼────────────┘
         │                    │
         ▼                    ▼
    Docker Socket        LLM Server
   (host mounted)     (external network)

The Overlord and Minions communicate over the nebulus-swarm Docker bridge network. The Overlord exposes port 8080 for external access (health checks, dashboard).

Production Considerations

Persistent State

Mount the state database to a persistent volume:

volumes:
  overlord-state:
    driver: local

Resource Limits

Minion containers are created with:

  • 2 GB memory limit
  • 1 CPU core
  • Auto-cleanup after exit

Monitoring

  • Health endpoint: GET /health returns 200 when Overlord is running
  • Status endpoint: GET /status returns active minions, config, pending questions
  • Docker health check: Configured in docker-compose with 30s intervals

Backup

Back up the SQLite state database regularly:

# Copy from Docker volume
docker cp overlord:/var/lib/overlord/state.db ./backup/state.db

# Or if using a mounted volume
cp /path/to/state.db ./backup/state.db

Log Management

Configure structured logging for production:

LOG_LEVEL=INFO
LOG_FORMAT=json
LOG_FILE=/var/log/overlord/overlord.log

Graceful Shutdown

The Overlord handles SIGTERM gracefully:

  1. Stops accepting new work
  2. Waits for active minions to finish (or timeout)
  3. Saves state to SQLite
  4. Exits cleanly
docker compose -f docker-compose.swarm.yml down

Overlord Daemon Deployment (Phase 3)

The Overlord daemon runs as a persistent background process with scheduled sweeps, Slack command routing, and proactive detection. It runs natively (not in Docker) and manages the cross-project ecosystem.

Prerequisites

  1. overlord.yml configured at ~/.atom/overlord.yml (see Configuration)
  2. Slack tokens set in environment (optional — daemon runs headless without them)
  3. croniter installed (pip install croniter)

Starting the Daemon

The daemon now auto-loads .env from the current working directory on startup. You no longer need to source .env manually.

# Via CLI (auto-loads .env from cwd)
cd /path/to/nebulus-atom
nebulus-atom overlord daemon start

# Output shows:
#   Loaded .env from /path/to/nebulus-atom/.env
#   SLACK_BOT_TOKEN: yes | SLACK_APP_TOKEN: yes | SLACK_CHANNEL_ID: yes
#   Logging to ~/.atom/overlord/daemon.log (level=INFO)
#   Starting Overlord daemon...

# Headless mode (scheduler only, no Slack — just don't set Slack vars)
nebulus-atom overlord daemon start

Logs are written to ~/.atom/overlord/daemon.log in structured JSON format. The log level defaults to INFO and can be overridden via the LOG_LEVEL environment variable.

Running as a Background Service

systemd (Linux)

# /etc/systemd/system/overlord-daemon.service
[Unit]
Description=Nebulus Overlord Daemon
After=network.target

[Service]
Type=simple
User=jlwestsr
WorkingDirectory=/home/jlwestsr/projects/west_ai_labs/nebulus-atom
EnvironmentFile=/home/jlwestsr/.atom/overlord.env
ExecStart=/home/jlwestsr/projects/west_ai_labs/nebulus-atom/venv/bin/python -m nebulus_atom.main overlord daemon start
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable overlord-daemon
sudo systemctl start overlord-daemon
sudo journalctl -u overlord-daemon -f

PM2 (macOS/Linux)

pm2 start "nebulus-atom overlord daemon start" --name overlord-daemon
pm2 save
pm2 startup

Daemon Architecture

OverlordDaemon
├── SlackBot (Socket Mode)        # Listens for @atom mentions
│   ├── SlackCommandRouter        # Routes to Phase 2 modules
│   └── ProposalManager           # Thread-based approval workflow
├── Startup Reconciliation        # Catches missed approve/deny replies
├── Scheduler Loop (croniter)     # Fires tasks on cron schedule
│   ├── scan (hourly)             # Health check + detection
│   ├── test-all (nightly)        # Test sweep
│   └── clean-stale-branches      # Weekly branch cleanup
├── Cleanup Loop (5 min)          # Expires stale proposals
├── DetectionEngine               # Stale, ahead-of-main, failing
├── NotificationManager           # Urgent + buffered digest
└── Signal Handler                # SIGINT/SIGTERM → graceful stop

Proposal Reconciliation on Startup

When the daemon starts with Slack integration, it performs a reconciliation sweep before entering the main loop. This catches approve/deny replies that were posted while the daemon was offline (Socket Mode does not buffer historical events).

How it works:

  1. After Socket Mode connects, the daemon queries all pending proposals with Slack threads
  2. For each proposal, it reads the thread history via the Slack API
  3. It scans replies (latest first) for approval keywords (approve, approved, yes, lgtm) or denial keywords (deny, denied, no, reject)
  4. Matching proposals are transitioned to APPROVED or DENIED
  5. A notification is posted in the thread explaining the reconciliation

Important limitation: DispatchPlans are cached in-memory and lost on restart. Reconciled approvals cannot auto-execute — the user is prompted to re-dispatch if the plan is still needed. Full plan persistence is a future enhancement.

Rate limiting: Proposals are processed in batches of 5 with a 1-second backoff between batches to avoid Slack API rate limits.

Stopping the Daemon

The daemon supports proper lifecycle management via the CLI:

# Check status
nebulus-atom overlord daemon status

# Stop gracefully (sends SIGTERM, waits up to 5s)
nebulus-atom overlord daemon stop

# Restart (stop + start)
nebulus-atom overlord daemon restart

# If running in foreground
Ctrl+C

# If running as a service
sudo systemctl stop overlord-daemon
# or
pm2 stop overlord-daemon

Graceful Shutdown

The daemon handles SIGINT and SIGTERM for clean shutdown:

  1. Sets the shutdown event
  2. Cancels scheduler, Slack bot, and cleanup tasks
  3. Waits for in-progress tasks to complete
  4. Stops the Slack bot connection
  5. Removes the PID file
  6. Logs shutdown complete

Persistent State

Data Location Description
PID file ~/.atom/overlord/daemon.pid Running daemon process ID
Daemon log ~/.atom/overlord/daemon.log Structured JSON daemon logs
Proposals DB ~/.atom/overlord/proposals.db Pending/completed proposals
Memory DB ~/.atom/overlord/memory.db Cross-project observations
State DB /var/lib/overlord/state.db Minion state (Phase 0)

Back up proposal and memory databases regularly:

cp ~/.atom/overlord/proposals.db ~/backup/
cp ~/.atom/overlord/memory.db ~/backup/

Monitoring the Daemon

# Check logs (systemd)
sudo journalctl -u overlord-daemon -f

# Check logs (PM2)
pm2 logs overlord-daemon

# Slack: ask the daemon directly
@atom status

The daemon logs all scheduled task executions, detection results, proposal state changes, and notification sends at INFO level.

Headless vs. Slack Mode

Feature Headless With Slack
Scheduled scans Yes Yes
Proactive detection Yes (logged) Yes (posted to Slack)
Slack commands No Yes
Proposal workflow No Yes (thread-based)
Urgent notifications No Yes
Daily digest No Yes
Proposal cleanup Yes Yes

Local Development

For development without Docker:

# Core agent
source .venv/bin/activate
python3 -m nebulus_atom.main start

# Swarm dashboard
streamlit run nebulus_swarm/dashboard/app.py

# Run tests
python3 -m pytest tests/ -v

Related Pages

Getting Started

Core System

Nebulus Swarm

Overlord Phase 3

Operations

Clone this wiki locally