Skip to content

LynkByte/ensemble

BranchesTags

Repository files navigation

ensemble-mcp

A harness infrastructure layer delivered as a Python MCP server — providing vector memory, drift detection, model routing, skills discovery, session management, codebase indexing, and context compression for AI agent pipelines.

All intelligence is local — zero LLM/API calls. Uses ONNX Runtime embeddings (~5ms), numpy cosine similarity, and SQLite storage.

What is an Agent Harness?

Agent = Model + Harness. A harness is every piece of code, configuration, and execution logic that wraps a model to turn it into a useful agent. Without a harness, a model can only take in text and output text — it can't maintain state, execute code, access tools, or learn from past work.

A complete agent harness typically includes:

  • System prompts & configuration — instructions that shape agent behavior
  • Tools, Skills & MCPs — capabilities the agent can invoke
  • Execution environment — filesystem, bash, sandbox, browser
  • Orchestration logic — subagent spawning, handoffs, model routing
  • Memory & context management — compaction, skills, session persistence
  • Hooks & middleware — deterministic checks like linting, formatting, drift detection

For a deeper dive, see The Anatomy of an Agent Harness by LangChain.

Where ensemble-mcp Fits

Agent harnesses like Claude Code, Codex, and Cursor provide the execution layer — filesystem, bash, sandbox. ensemble-mcp provides the intelligence infrastructure layer that plugs into any harness via MCP:

┌─────────────────────────────────────────────────────────┐
│                    AGENT HARNESS                        │
│                                                         │
│  ┌───────────────────────────────────────────────────┐  │
│  │            Execution Layer                        │  │
│  │    (provided by Claude Code / Codex / Cursor)     │  │
│  │                                                   │  │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────────────┐  │  │
│  │  │Filesystem│ │  Bash /  │ │   Sandbox /      │  │  │
│  │  │  & Git   │ │  Code    │ │   Browser        │  │  │
│  │  └──────────┘ └──────────┘ └──────────────────┘  │  │
│  └───────────────────────────────────────────────────┘  │
│                                                         │
│  ┌───────────────────────────────────────────────────┐  │
│  │       Intelligence Infrastructure Layer           │  │
│  │         (provided by ensemble-mcp via MCP)        │  │
│  │                                                   │  │
│  │  ┌────────┐ ┌────────┐ ┌───────┐ ┌───────────┐  │  │
│  │  │ Memory │ │ Skills │ │ Drift │ │  Context  │  │  │
│  │  │& Search│ │        │ │Detect │ │Compression│  │  │
│  │  └────────┘ └────────┘ └───────┘ └───────────┘  │  │
│  │  ┌────────┐ ┌────────┐ ┌───────┐ ┌───────────┐  │  │
│  │  │ Model  │ │Session │ │Code   │ │ Project   │  │  │
│  │  │Routing │ │Persist.│ │Index  │ │ Snapshot  │  │  │
│  │  └────────┘ └────────┘ └───────┘ └───────────┘  │  │
│  └───────────────────────────────────────────────────┘  │
│                                                         │
│  ┌───────────────────────────────────────────────────┐  │
│  │          Orchestration Layer                       │  │
│  │   (agent pipeline, subagent spawning, handoffs)   │  │
│  └───────────────────────────────────────────────────┘  │
│                                                         │
│                    ┌───────────┐                        │
│                    │   MODEL   │                        │
│                    └───────────┘                        │
└─────────────────────────────────────────────────────────┘

Features

ensemble-mcp provides these harness primitives:

Harness Primitive Feature What It Does
Memory & Search Pattern Memory Semantic vector search over stored pipeline patterns (MiniLM-L6-v2, 384-dim)
Drift Detection Drift Check Cosine similarity between task description and code changes
Model Routing Model Recommend Recommend model tier (best/mid/cheapest) per agent and task complexity
Skills (Progressive Disclosure) Skills Discovery Scan .ai/skills/, .claude/skills/, .cursor/rules/ etc. with semantic search
Skills (Progressive Disclosure) Skill Intelligence Auto-detect recurring patterns and suggest converting them to reusable skills
Long Horizon Execution Session Management Pipeline checkpoint save/load with optimistic versioning
Codebase Awareness Codebase Indexing File-level index with exports, imports, roles — incremental via mtime
Context Rot Prevention Context Compression Compress verbose text into token-efficient form, rule-based
Context Rot Prevention Prompt Caching Order and prepare prompt sections for optimal LLM cache hit rates
Codebase Awareness Codebase Snapshots Generate compact project baseline summaries from the codebase index
Harness Setup Auto-Installer Detect AI tools and register the MCP server in their configs
Observability Web Dashboard Local browser UI at localhost:8787 for visualizing patterns, skills, projects, drift, and sessions

Quick Start

Install from source

git clone https://github.com/LynkByte/ensemble.git
cd ensemble
pip install -e ".[dev]"

Install via uvx (after PyPI publish)

uvx ensemble-mcp

Run the server

# Start the MCP server (stdio protocol)
ensemble-mcp

# Or explicitly
ensemble-mcp serve

Other CLI commands

# Auto-detect AI tools and register the MCP server
ensemble-mcp install

# Remove MCP server registration from AI tool configs
ensemble-mcp uninstall

# Copy agent files to a project (no MCP registration needed)
ensemble-mcp add-agents --tools opencode

# Copy skill files to a project (no MCP registration needed)
ensemble-mcp add-skills --tools opencode

Web Dashboard

A local-only browser dashboard for visualizing patterns, skills, projects, drift history, and sessions.

# Start the dashboard (opens browser to http://localhost:8787)
ensemble-mcp web

# Custom port
ensemble-mcp web --port 9000

# Start without auto-opening the browser
ensemble-mcp web --no-open

# Specify a reports directory (auto-detected from ./reports or git root otherwise)
ensemble-mcp web --reports-dir /path/to/reports

The dashboard reads directly from the same SQLite database the MCP server writes to (WAL mode, no contention). It binds to 127.0.0.1 only — never exposed to the network, no authentication needed.

Dashboard Pages

Page What It Shows
Overview Summary cards (patterns, skills, projects, drift checks), drift score trend chart, recent activity
Patterns All stored patterns with match counts, project filtering, search
Skills Pending skill suggestions with confidence scores, stale skill detection
Projects Indexed projects with language breakdown pie charts, file role bar charts, export counts
Drift Drift check history with scores, verdicts, flagged files, project filtering
Sessions Session list with lifecycle status, click-through to step-by-step detail

API Endpoints

All endpoints return the standard ok/data/error/meta envelope:

Endpoint Description
GET /api/summary Aggregate counts and recent activity
GET /api/patterns Paginated pattern list (filter by project)
GET /api/patterns/:id Single pattern detail with embedding metadata
GET /api/skills Skill suggestions (filter by project, status)
GET /api/skills/stale Skills not matched within threshold days
GET /api/projects Indexed projects with file/export counts
GET /api/projects/:path Project detail with language and role breakdown
GET /api/drift Drift history (filter by project, from, to)
GET /api/sessions Paginated session list (filter by project, status)
GET /api/sessions/:id Session detail with steps
GET /api/health Server health, version, DB size

MCP Client Configuration

OpenCode

Add to ~/.config/opencode/config.json or project config.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "ensemble": {
      "type": "local",
      "command": ["uvx", "ensemble-mcp"]
    }
  }
}

Or for a local development install:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "ensemble": {
      "type": "local",
      "command": ["/path/to/venv/bin/ensemble-mcp"]
    }
  }
}

Claude Code

Add to ~/.claude/claude_desktop_config.json:

{
  "mcpServers": {
    "ensemble": {
      "command": "uvx",
      "args": ["ensemble-mcp"]
    }
  }
}

GitHub Copilot (VS Code)

Add to .vscode/mcp.json:

{
  "servers": {
    "ensemble": {
      "command": "uvx",
      "args": ["ensemble-mcp"]
    }
  }
}

Cursor

Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "ensemble": {
      "command": "uvx",
      "args": ["ensemble-mcp"]
    }
  }
}

Windsurf

Add to ~/.windsurf/mcp.json:

{
  "mcpServers": {
    "ensemble": {
      "command": "uvx",
      "args": ["ensemble-mcp"]
    }
  }
}

Auto-Install

Instead of manual configuration, run the installer to auto-detect installed AI tools and register the server:

# Detect tools and register (interactive)
ensemble-mcp install

# Register specific tools only
ensemble-mcp install --tools opencode,cursor

# Preview without making changes
ensemble-mcp install --dry-run

# Non-interactive
ensemble-mcp install --yes

The installer automatically detects how ensemble-mcp is available on your system and registers the appropriate command:

Priority Detection Registered Command
1st ensemble-mcp on PATH (pip/pipx install) ensemble-mcp
2nd uvx on PATH uvx ensemble-mcp
3rd Neither found /path/to/python -m ensemble_mcp (current interpreter)

19 MCP Tools

Patterns (semantic memory)

Tool Description
patterns_search Semantic search over stored patterns
patterns_store Store a new pattern with embedding
patterns_prune Remove old/unused patterns

Drift Detection

Tool Description
drift_check Cosine similarity between task and changes (0-1 score)

Model Routing

Tool Description
model_recommend Recommend model tier for agent + task complexity

Skills

Tool Description
skills_discover Scan skill directories with optional semantic search
skills_suggest Detect recurring patterns, propose as reusable skills
skills_generate Accept, dismiss, or defer a skill suggestion

Session

Tool Description
session_save Save pipeline checkpoint with optimistic versioning
session_load Load latest or specific checkpoint
session_search Search sessions by semantic similarity

Codebase Indexer

Tool Description
project_index Build/refresh file-level codebase index
project_query Query index by language, path, or text
project_dependencies Get import/dependency graph for a file
project_snapshot Generate compact project baseline summary (cached)

Context Compression

Tool Description
context_compress Compress verbose text into terse, token-efficient form while preserving technical content
context_prepare Order prompt sections for optimal LLM cache hit rates (static → project → task)

Utility

Tool Description
health Server health check
reset Reset all data (destructive, requires confirmation)

Response Envelope

Every tool returns a standardized envelope:

{
  "ok": true,
  "data": { "matches": [...] },
  "error": null,
  "meta": {
    "duration_ms": 12,
    "source": "sqlite",
    "confidence": "exact"
  }
}

Confidence indicators: exact (direct data), partial (mixed sources), estimated (heuristic).

Architecture

ensemble-mcp is structured as a harness infrastructure layer with clear separation between tool implementations and shared engine components:

ensemble-mcp/
  src/ensemble_mcp/
    server.py             # MCP server + tool registration
    config/               # Settings, defaults
    contracts/            # Response envelope, error taxonomy
    memory/               # ONNX embeddings, SQLite vector store, cosine similarity
    security/             # Secret redaction, trust boundaries
    state/                # Session/step lifecycle, idempotency, locks
    tools/                # 19 MCP tool implementations + call-recording utility
    installer/            # AI tool detection + MCP registration
    dashboard/            # Web dashboard (aiohttp server, JSON API, SPA frontend)
    compress/             # Rule-based text compression engine (context rot prevention)
    cli/                  # Startup banner
    data/                 # Bundled agent and skill files (AGENTS.md, skills)

Technology Stack

Component Choice
Language Python 3.11+
Distribution uvx (zero-hassle cross-platform)
MCP Framework mcp (official Python SDK)
Embeddings ONNX Runtime + MiniLM-L6-v2 (~22MB, 384-dim)
Vector Storage SQLite + numpy cosine similarity
Tokenizer HuggingFace tokenizers (for MiniLM input)
Package Size ~90MB (including ONNX + model)

Local Storage

Path Contents
~/.cache/ensemble-mcp/data.db SQLite database (WAL mode)
~/.cache/ensemble-mcp/models/ ONNX MiniLM-L6-v2 model (~22MB)
~/.config/ensemble-mcp/config.toml Global user configuration

Configuration

Layered config with deterministic merge order:

  1. Package defaults (built-in)
  2. Global config (~/.config/ensemble-mcp/config.toml)
  3. Project config (.ensemble-mcp.toml)
  4. Environment variables (ENSEMBLE_MCP_*)
# ~/.config/ensemble-mcp/config.toml
drift_threshold_aligned = 0.25
default_top_k = 5
cluster_similarity_threshold = 0.8

Development

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
python -m pytest tests/ -v

# Lint
ruff check src/ tests/

# Format
ruff format src/ tests/

# Type check
mypy src/

# Build package
python -m build

Docker

docker build -t ensemble-mcp .
docker run --rm -v ~/.cache/ensemble-mcp:/home/app/.cache/ensemble-mcp ensemble-mcp

Eval Framework

Benchmark suite for measuring tool effectiveness:

# Run eval benchmarks
python evals/runner.py

# Or via pytest
python -m pytest evals/ -v

Supported AI Tools

AI Tool Config Format Auto-Install
OpenCode JSON Yes
Claude Code JSON Yes
GitHub Copilot (VS Code) JSON Yes
Cursor JSON Yes
Windsurf JSON Yes
Devin CLI JSON Yes

License

MIT

About

Comprehensive system design for improving the Ensemble multi-agent orchestration system and building a companion Python MCP server.

Resources

Readme

License

MIT license

Contributing

Contributing
Activity
Custom properties

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

15 tags

Packages

 
 
 

Contributors

Languages