CodebaseRAG

Local-first semantic search for your codebase via MCP (Model Context Protocol)

A personal cognitive weapon that lets GitHub Copilot (or any MCP-compatible LLM) search your entire codebase semantically — without uploading a single file to the cloud.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    YOUR CODEBASE (Local)                    │
│   *.ts, *.tsx, *.js, *.py, *.md, *.json, *.yaml, *.sql     │
└──────────────────────────────┬──────────────────────────────┘
                               │ chokidar (file watcher)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                    INDEXER (on-save)                        │
│   1. Read file → 2. Smart chunk → 3. Embed → 4. Store      │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                 LANCEDB (.rag_index/)                       │
│        Embedded vector store (SQLite-backed, zero-config)   │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                    MCP SERVER (stdio)                       │
│        Tool: search_codebase(query) → top-7 chunks          │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                   VS CODE + COPILOT                         │
│        Asks questions → Gets grounded answers               │
└─────────────────────────────────────────────────────────────┘

Components

Component	Role	Tool Used
Librarian	Generates embeddings (dumb but fast)	Ollama + nomic-embed-text
Memory	Stores vectors locally	LanceDB
Translator	Exposes search_codebase tool to LLMs	MCP Server
Brain	Answers questions using retrieved context	Copilot/Claude/GPT

Prerequisites

1. Node.js 20+ - node --version
2. Ollama - Install Ollama
3. Embedding model - Run: ollama pull nomic-embed-text

Installation

git clone <this-repo>
cd CodebaseRAG
npm install

Configuration

Add a workspace-scoped config at .vscode/mcp.json:

{
  "servers": {
    "CodebaseRAG": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "tsx",
        "C:/path/to/CodebaseRAG/server.ts"
      ],
      "env": {
        "RAG_PROJECT_ROOT": "${workspaceFolder}"
      }
    }
  }
}

Note: The server prefers MCP-provided workspace roots. The env override is a fallback.

Usage

CLI Options

# Specify project root explicitly
npx tsx server.ts --root /path/to/project

# Or via environment variable
RAG_PROJECT_ROOT=/path/to/project npx tsx server.ts

# Force reset (wipe and rebuild index)
npx tsx server.ts --root /path/to/project --reset

# Or via environment variable
RAG_RESET=true RAG_PROJECT_ROOT=/path/to/project npx tsx server.ts

Automatic (VS Code + Copilot)

1. Open any workspace
2. Ask Copilot: "How does authentication work in this repo?"
3. Copilot will invoke search_codebase and answer using actual code

Safety Features

Strict Project Scoping

• Workspace root comes from MCP (preferred)
• Explicit overrides supported via --root or RAG_PROJECT_ROOT
• process.cwd() is NEVER trusted blindly
• Initialization fails if no safe root is provided

Dangerous Path Protection

Automatically blocks dangerous roots:

• /, C:\, D:\ (drive roots)
• /home, /Users, C:\Users (user directories)
• Home directory and common subfolders

Hard Ignore Boundaries

Never indexed, even if inside project:

• node_modules/, .git/, dist/, build/
• .next/, .turbo/, .cache/
• The vector DB directory itself (.rag_index/)
• Binary files, symlinks (not followed)

How It Works

On Startup

1. MCP handshake completes
2. Resolves workspace root (MCP roots → explicit override)
3. Validates root (exists, not dangerous)
4. Checks for reset flag
5. Initializes LanceDB
6. Scans and indexes scoped project
7. Starts file watcher

On File Save

1. File watcher detects change
2. Deletes old vectors for that file
3. Re-embeds only the changed file
4. Updates the vector store

On Query

1. Embeds your natural language query
2. Performs vector similarity search
3. Returns top 7 most relevant code chunks

Smart Chunking

• Code files (.ts, .js, .py): Splits by function/class boundaries
• Prose files (.md, .json): Splits by paragraphs/sections
• Chunk size: 50-2000 characters (optimal for embedding quality)

Files Indexed

*.ts, *.tsx, *.js, *.jsx, *.py, *.md, *.json, *.yaml, *.yml, *.sql, *.sh, *.css, *.html

Files Ignored

node_modules/, dist/, build/, out/, .git/, .next/, .turbo/, .cache/,
__pycache__/, *.lock, *.min.js, *.min.css, .env*, *.log, coverage/

Resource Usage

Resource	Estimate
RAM	~200-400MB (Ollama) + ~50MB (Node process)
Disk	~1-5MB per 1000 files (LanceDB)
CPU	Minimal idle; spikes during indexing
Latency	<300ms per query (local)

Tool Schema

search_codebase(query: string): string

Input:

• query: Natural language question or code snippet

Output:

• Top 7 relevant code chunks with file paths

Example Queries

"How does the user authentication flow work?"
"Where is the database connection configured?"
"Find all API error handlers"
"What environment variables are used?"
"Show me the UserService class"

Clearing / Resetting the Index

# Option 1: CLI flag
npx tsx server.ts --root /path/to/project --reset

# Option 2: Environment variable
RAG_RESET=true npx tsx server.ts --root /path/to/project

# Option 3: Manual deletion
rm -rf /path/to/project/.rag_index

Troubleshooting

"FATAL: No project root specified"

You must provide an explicit root:

npx tsx server.ts --root /path/to/project

"FATAL: Dangerous project root detected"

The path you specified is too broad (e.g., C:\ or home directory). Specify a specific project folder.

"Cannot connect to Ollama"

# Ensure Ollama is running
ollama serve

"Model not found"

# Pull the embedding model
ollama pull nomic-embed-text

"No results found"

• Check if .rag_index/ folder exists in your project
• Run with --reset to rebuild the index
• Try more specific queries

Directory Structure

CodebaseRAG/
├── server.ts          # MCP server + indexer
├── package.json       # Dependencies
├── tsconfig.json      # TypeScript config
├── README.md          # This file
└── .gitignore         # Ignore rules

In your project:

your-project/
├── .rag_index/        # LanceDB storage (auto-created)
└── ... your code ...

Stop Point (Definition of Done)

This system is COMPLETE when:

• Project root explicitly required (no blind cwd trust)
• Dangerous paths blocked automatically
• Reset mechanism via --reset or RAG_RESET
• File watcher scoped to project only
• Symlinks not followed
• DB directory ignored by watcher
• Startup logs resolved root and file count
• RAM under 500MB
• Query latency under 300ms

---

No UI. No auth. No cloud. No complexity.

Just you, your code, and an LLM that actually knows what's in your repo.