SpecSearch

Hybrid RAG search engine over construction-spec PDFs.

Six-stage pipeline: ingest → dual-index (Elasticsearch BM25 + ChromaDB vectors) → parallel retrieve → RRF fusion → cross-encoder rerank → Ollama cited answer.

Quickstart

cp .env.example .env          # edit OLLAMA_MODEL if needed
docker compose up -d elasticsearch
uv sync
uv run specsearch index data/raw
uv run specsearch query "What is the required concrete compressive strength at 28 days?" --explain

Surfaces

Surface	Command
CLI	uv run specsearch query "..."
Streamlit UI	uv run streamlit run src/specsearch/app.py
MCP (Claude Desktop)	see mcp_server.py setup below

RESULTS

Config	Recall@10	MRR	nDCG@10
BM25 only	1.000	0.792	0.853
Vector only	0.438	0.400	0.375
Hybrid (RRF)	0.688	0.466	0.484
Hybrid + rerank	0.438	0.396	0.371

Stage	median ms	p95 ms
retrieve_fuse	55.0	58.0
rerank	1534.5	1578.0
generate	3869.0	4635.0
total	5429.5	6239.0

Design

See docs/DESIGN.md for architecture rationale and decision log.

Roadmap

Retrieval & indexing

• PDF table extraction for mix-proportion and strength-class data currently missed by text-only parsing
• Incremental re-indexing without a full rebuild (document hash tracking)
• Domain-fine-tuned cross-encoder on construction query/passage pairs
• CSI code query expansion ("concrete durability" searches all "03 xx xx" sections)
• Spec diff mode -- surface changed requirements between two spec versions

Generation

• Engineered system prompt with few-shot citation examples and domain-specific grounding instructions
• Streaming answers (SSE) for real-time token display

Surfaces & integrations

• Next.js web UI with side-by-side source PDF viewer and highlighted citation passages
• Expanded MCP surface: list_specs, get_section, compare_specs, and diff_specs tools alongside the existing search_specs
• Full Docker Compose stack including Ollama for one-command local setup

MCP — Claude Desktop setup

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "specsearch": {
      "command": "uv",
      "args": ["run", "python", "-m", "specsearch.mcp_server"],
      "cwd": "/absolute/path/to/specsearch"
    }
  }
}

Replace /absolute/path/to/specsearch with the actual repo path, then restart Claude Desktop. You can then ask Claude: "Search the specs for the required concrete compressive strength."