Sift

A semantic aggregator for MCP tools. Point Sift at every MCP server you have — it exposes exactly two tools to your agent, no matter how many hundreds are aggregated behind them.

---

The problem

MCP made it trivial to give an agent a tool server. It did not solve what happens once you give it ten of them.

Every MCP client dumps the full tools/list of every connected server straight into the model's context on every turn — name, description, and JSON schema, all of it, every time. That doesn't stay cheap:

• The context bill keeps growing. Fifteen servers with a dozen tools each is 150+ full tool definitions loaded before the model has read a single word of the actual task — tokens spent every turn on tools that turn won't use.
• Tool selection gets worse, not better, as you add tools. Buried in a wall of search_docs_v2, search_documents, docsSearch, the model starts guessing. More tools does not mean better coverage; past a point it means more misfires.
• Every server is a hard dependency. One slow or half-configured downstream server can stall tools/list for the whole session, even for tools nobody's calling.
• None of it survives a restart. Tool catalogs are typically rebuilt from scratch on boot — reconnecting to every server and re-describing every tool before the agent can do anything.

None of this is a flaw in any one MCP server. It's what happens when the "list everything, always" model that works for three tools is pointed at thirty.

The approach

Sift sits between your agent and every downstream MCP server, and inverts the model: instead of the agent seeing every tool from every server, it sees two:

• search_tools — describe what you're trying to do in plain language, get back the handful of tools (across all aggregated servers) that actually match, ranked by relevance.
• call_tool — invoke one of them by the tool_id search_tools returned.

The full catalog still exists — searchable, callable, semantically indexed — it just never has to sit in the model's context wholesale. Context cost stops scaling with how many servers you've aggregated and starts scaling with how many tools a given task actually needs, which is almost always a small, constant number.

Everything downstream of that search is real infrastructure, not a demo shortcut:

• Local and remote tools in one catalog. Hand-registered Rust functions and tools proxied from downstream MCP servers (stdio or Streamable HTTP) are searched and called through the identical path.
• Persisted, not rebuilt. Tool descriptions and their embeddings are written through to SQLite as they're learned. A restart reloads the catalog — and search works again — without recomputing a single embedding or reconnecting to a single server first.
• Reconciled, not replaced. Re-listing a downstream server's tools diffs against what's already known: unchanged tools are left alone, changed ones are re-embedded, removed ones are tombstoned. A resync costs work proportional to what actually changed.
• A server going down degrades, it doesn't cascade. A tool whose server hasn't reconnected yet reports itself unavailable on call — it doesn't block search, and it doesn't take the other servers with it.

How it fits together

              ┌────────────────────────────────────────────┐
  agent /     │                    sift                     │
  MCP client  │                                              │
  ─────────►  │  search_tools ─┐                             │
              │  call_tool   ──┼──►  ToolRegistry             │
              │                │      • semantic index        │      downstream MCP servers
              │                │        (fastembed, in-mem)   │  ┌─► local Rust fn
              │                │      • SQLite catalog         │──┤
              │                │        (survives restarts)   │  ├─► stdio server
              │                └──►  reconcile on              │  └─► streamable-http server
              │                     onboard / resync           │
              └────────────────────────────────────────────┘

Sift is itself a Streamable HTTP MCP server (POST /mcp) — any MCP client connects to it exactly like it would to any single downstream server, and never has to know how many are actually behind it. A plain REST API (/tools/search, /tools/call) is exposed alongside it for testing and for clients that don't speak MCP.

Features

• Semantic tool search. search_tools embeds the query and ranks every aggregated tool by cosine similarity (fastembed, quantized BGE-small, 384-dim) — no exact tool name required, and it works the same whether there are 5 tools or 5,000.
• One catalog, mixed sources. Hand-registered local Rust functions and tools proxied from downstream MCP servers over stdio or Streamable HTTP are searched and called through the same tool_id scheme ({server}::{name}).
• Two protocols, same registry. Talk to Sift as an MCP server itself (POST /mcp — initialize, tools/list, tools/call) or as plain REST (/tools/search, /tools/call); both hit the identical ToolRegistry.
• Durable catalog. Every tool write-through's to SQLite as it's learned, so a restart reloads the full searchable catalog — embeddings included — without re-embedding anything or waiting on a downstream server to reconnect.
• Incremental reconciliation. Re-listing a server's tools diffs against what's already known: unchanged tools are untouched, changed ones are re-embedded, removed ones are tombstoned — a resync costs work proportional to what actually changed, not the size of the catalog.
• Isolated failures. Every downstream server onboards independently and a per-call timeout guards every invocation, so one slow or disconnected server can't stall search or calls against the rest of the catalog.
• Schema-validated calls. Arguments are checked against the tool's declared JSON Schema before it's ever invoked, with structured error kinds (not found, pending reconnect, bad schema, timeout, handler failure) mapped to sane HTTP status codes on the REST API.
• Lock-free concurrent reads. The search index is an ArcSwap snapshot over a DashMap catalog — concurrent searches and calls never block a registration or a resync in progress.
• Feature-gated builds. sqlite, stdio-transport, and http-transport are independent Cargo features, all on by default, so a build can be trimmed to exactly the transports and persistence it needs.

Quickstart

Docker Compose

Requires Docker with the Compose plugin (docker compose, not the standalone docker-compose). No local Rust toolchain needed — everything, including the embedding model, is baked into the image at build time.

docker compose up --build

This builds one image (sift-aggregator:latest, see Dockerfile) and starts two containers from it:

• mcp-sample — the sample downstream MCP server, on http://localhost:8090/mcp (real tools: add, search_docs).
• aggregator — Sift itself, configured via config.docker.toml to onboard mcp-sample by its Compose service name. Exposed on host port 8088 → container port 8080 (chosen because 8080 is often already taken locally, e.g. by k3d).

Wait for listening in the aggregator's logs, then try it:

curl -s localhost:8088/healthz

curl -s localhost:8088/tools/search -d '{"query":"add two numbers"}' | jq

curl -s localhost:8088/tools/call -d '{
  "tool_id": "sample-http::add",
  "arguments": {"a": 2, "b": 3}
}' | jq

Useful while it's running:

docker compose logs -f aggregator   # tail just the aggregator's logs
docker compose ps                   # container status
docker compose down                 # stop and remove both containers

Re-run docker compose up --build after any code change — Compose reuses the cached image layers, so only what actually changed gets rebuilt.

From source

Both processes at once:

./scripts/run_demo.sh

This builds and runs the sample downstream MCP server plus the aggregator (wired together via config.toml), then prints ready-to-run curl examples for both the plain REST API and the aggregator's own /mcp endpoint.

# search across every aggregated server by intent, not by name
curl -s localhost:8080/tools/search -d '{"query":"add two numbers"}' | jq

# call the tool_id search_tools returned
curl -s localhost:8080/tools/call -d '{
  "tool_id": "sample-http::add",
  "arguments": {"a": 2, "b": 3}
}' | jq

# ...or speak MCP directly — this is what a real agent sees
curl -s localhost:8080/mcp -d '{
  "jsonrpc": "2.0", "id": 1, "method": "tools/call",
  "params": {"name": "search_tools", "arguments": {"query": "add two numbers"}}
}' | jq

cargo run -p aggregator-core --release --example demo runs the same idea in-process — local tools and a real downstream server side by side, searched and called by plain-language query — without standing up the HTTP server at all.

Configuring downstream servers

Servers to aggregate are declared in config.toml (override the path with AGGREGATOR_CONFIG_PATH):

[[servers]]
server_id = "sample-http"
kind = "streamable_http"
url = "http://localhost:8090/mcp"
enabled = true

[[servers]]
server_id = "fs"
kind = "stdio"
command = "mcp-fs"
args = ["--root", "/data"]

Sift connects to every enabled server concurrently on boot and reconciles its live tool list into the catalog; one server failing to connect never blocks the others.

API

Endpoint	Method	Purpose
/healthz	GET	Liveness check
/tools/search	POST	{query, k?} → ranked tool matches across every aggregated server
/tools/call	POST	{tool_id, arguments} → invoke a tool by the id search returned
/mcp	POST	Sift as an MCP server — initialize, tools/list (search_tools, call_tool), tools/call

Building

cargo build --workspace --all-features
cargo test --workspace --all-features
cargo clippy --workspace --all-features -- -D warnings

Feature flags on aggregator-core (all on by default):

Feature	Enables
sqlite	Write-through SQLite persistence for the tool catalog
stdio-transport	Aggregating downstream servers over stdio
http-transport	Aggregating downstream servers over Streamable HTTP

Project layout

aggregator-core/    ToolRegistry, embedding engine, SQLite store, MCP client transports
aggregator-server/  Axum HTTP server: REST API + Sift's own /mcp endpoint
scripts/run_demo.sh Runs the sample downstream server + aggregator together