Testing

Testing

Nebulus Atom uses pytest with pytest-asyncio for testing. The test suite contains 1219 tests covering core functionality, swarm orchestration, Overlord meta-orchestration, and integrations.

Running Tests

# Run all tests
python3 -m pytest tests/ -v

# Run a specific test file
python3 -m pytest tests/test_model_router.py -v

# Run a specific test class
python3 -m pytest tests/test_model_router.py::TestComplexityAnalysis -v

# Run a specific test
python3 -m pytest tests/test_model_router.py::TestComplexityAnalysis::test_empty_issue_scores_zero -v

# Run with output capture disabled (see print statements)
python3 -m pytest tests/ -v -s

Test Organization

Core Agent Tests

File	Tests	Coverage
test_context_manager.py	Context pinning and unpinning
test_smart_undo.py	Checkpoint creation and restoration
test_skill_library.py	Skill CRUD and execution
test_rag.py	RAG service and embeddings
test_autonomous_execution.py	Auto-mode execution loop
test_tdd_loop.py	TDD cycle automation
test_tool_registry.py	Tool deduplication and listing
test_response_parser.py	JSON extraction from LLM output
test_cognition_service.py	Advanced reasoning
test_cli_features.py	CLI command features
test_interactive.py	Interactive clarification
test_mcp.py	MCP client integration
test_multimodal.py	Multimodal input handling
test_ast_service.py	Code analysis
test_sandbox.py	Sandboxed execution
test_logging.py	Structured logging
test_journal_service.py	Session journal
test_macro_service.py	Shell macro generation
test_preference_learning.py	Adaptive preferences
test_task_persistence.py	Task state persistence
test_doc_service.py	Embedded documentation

Swarm Tests

File	Tests	Coverage
test_swarm.py	Overlord, DockerManager, State, GitHubQueue
test_minion.py	Minion lifecycle, Reporter, GitManager
test_minion_agent.py	MinionAgent, ToolExecutor, LLMClient
test_llm_parser.py	LLM command parsing, context store
test_minion_question.py	Clarifying questions flow (26 tests)
test_swarm_dashboard.py	Dashboard data client, state queries (22 tests)
test_model_router.py	Complexity analysis, model selection (32 tests)

Overlord Tests (393 tests)

File	Tests	Coverage
test_overlord_registry.py	28	Config loading, YAML parsing, dependency order, validation
test_overlord_scanner.py	18	Git state scanning, test health, branch analysis
test_overlord_graph.py	22	DAG traversal, upstream/downstream, subgraph, ASCII render
test_overlord_action_scope.py	20	Blast radius, scope evaluation, autonomy scoring
test_overlord_memory.py	22	SQLite store, remember/search/forget/prune
test_overlord_autonomy.py	24	Autonomy engine, confidence scoring, pre-approved actions
test_overlord_dispatch.py	25	Multi-repo dispatch, plan execution, rollback
test_overlord_model_router.py	18	Complexity analysis, tier selection
test_overlord_release.py	20	Release coordination, dependency-order execution
test_overlord_task_parser.py	15	Natural language → DispatchPlan conversion
test_overlord_e2e.py	8	Phase 2 end-to-end integration
test_overlord_slack_commands.py	31	Slack command parsing, handlers, async wrapping, formatting
test_overlord_proposal_manager.py	23	Proposal lifecycle, SQLite store, thread mapping, expiry
test_overlord_daemon.py	25	Daemon lifecycle, scheduler, task execution, graceful shutdown
test_overlord_detectors.py	24	Stale branch, ahead-of-main, failing test detection
test_overlord_notifications.py	19	Urgent alerts, buffered accumulation, daily digest
test_overlord_phase3_e2e.py	10	Full Phase 3 lifecycle: daemon → detect → propose → approve → notify
test_overlord_cli.py	21	CLI command registration and output

Agent Harness Tests

Integration-style tests in tests/agent_harness/:

File	Coverage
test_core_capabilities.py	File creation, context resilience, JSON stability
test_telemetry.py	Event logging, tool executor logging
test_recovery.py	Error interception and recovery
test_truncation.py	Response truncation handling
test_rag.py	RAG tool exposure and dispatch
test_skills.py	Skill creation and execution
test_cognition.py	JSON extraction with reasoning
test_ui_launch.py	Dashboard launch verification

Deployment Tests

File	Coverage
tests/deployment/test_sandbox.py	Sandbox write permissions

Testing Patterns

Mocking External Dependencies

The test suite mocks several external dependencies:

slack_bolt (not installed in dev):

import sys
from unittest.mock import MagicMock

sys.modules.setdefault("slack_bolt", MagicMock())
sys.modules.setdefault("slack_bolt.adapter.socket_mode.async_handler", MagicMock())
sys.modules.setdefault("slack_bolt.async_app", MagicMock())

Docker/GitHub (no real connections in tests):

from unittest.mock import patch, MagicMock

@patch("nebulus_swarm.overlord.docker_manager.docker")
def test_spawn(self, mock_docker):
    ...

Async Tests

Tests using async def require the @pytest.mark.asyncio decorator:

@pytest.mark.asyncio
async def test_async_handler(self):
    result = await handler(request)
    assert result.status == 200

Temporary Files

Use tmp_path fixture for tests needing file system:

def test_state_db(self, tmp_path):
    db_path = str(tmp_path / "test.db")
    state = OverlordState(db_path=db_path)
    ...

Pre-commit Hooks

The project enforces code quality via pre-commit:

• ruff: Linting (catches unused imports, ambiguous names, etc.)
• ruff-format: Code formatting
• trailing whitespace: Removes trailing spaces
• end-of-file-fixer: Ensures files end with newline

# Install hooks
pre-commit install

# Run manually
pre-commit run --all-files

Adding Tests

When adding new features:

1. Create a test file: tests/test_<feature>.py
2. Use descriptive test class and method names
3. Mock external dependencies
4. Use tmp_path for file system tests
5. Run the full suite before committing: python3 -m pytest tests/ -v

Related Pages

• Architecture - System structure
• Contributing - Development workflow