Code-level RAG: using Claude Code with a custom knowledge base

Note: This project is a reference implementation, not a plug-and-play tool. It was built for a specific monorepo and workflow, so it won't work out of the box. The recommended approach is to feed this repo into an LLM (like Claude), have it review the architecture and ideas, and then generate a version tailored to your own project structure and tooling.

The problem

AI coding assistants forget everything between sessions. Every new conversation starts from zero with no memory of past decisions, debugging sessions, or architectural choices. On a large codebase, you end up re-explaining context and watching the AI rediscover things you already solved last week.

I wanted Claude Code to learn from every PR I ship.

The solution

I built a RAG pipeline that captures the full story of every code change (the plan, the conversation log, the summary) and makes it searchable through vector embeddings. When I start new work, Claude Code queries this knowledge base for relevant past implementations before writing a line of code.

The system has about 450 indexed documents so far.

How ticket tracking evolved

The system started with Linear for project management, connected through a custom UTCP (Universal Tool Calling Protocol) HTTP bridge. UTCP exposed Linear's API as callable tools (linear_get_issue, linear_list_issues, linear_update_issue, etc.) so Claude Code could fetch ticket details and update statuses during PR workflows.

Reading tickets worked fine. Status updates didn't. Marking a ticket "Done" programmatically would silently fail, which meant I still had to go into the Linear UI manually. For a workflow that depends on automation at PR close, that killed the whole point.

I switched to Notion in early February 2026. Notion's MCP integration handles property updates reliably, so the PR close command marks tickets Done every time without intervention. It also consolidated tooling since I was already using Notion for other project docs.

The UTCP bridge still handles Gmail, but ticket management moved entirely to Notion MCP.

How it works

1. Slash commands run the PR lifecycle

The workflow runs through Claude Code custom commands (slash commands).

/pr-start <branch-name> <folder> kicks off a new PR:

• Creates an isolated git worktree (separate directory, separate branch, shared history)
• Runs pnpm install in the worktree
• Creates the changelog folder with empty PLAN.md and CHANGELOG.md
• Writes .claude/current-project.txt so hooks know which PR is active
• Copies MCP server config to the worktree so Notion/Supabase tools work there
• Extracts the ticket ID from the branch name, fetches the Notion ticket, and populates PLAN.md with the title, description, and action items

After that, I open a new Claude Code session in the worktree directory and start working.

/pr-close wraps up the PR in sequence:

1. Generates pr-summary-{date}.md with an overview, changes, and code examples
2. Condenses CHANGELOG.md and PLAN.md (summarizes verbose content, keeps timestamps)
3. Runs pnpm kb:ingest to index the changelog files into Supabase
4. Updates the Notion ticket status to Done
5. Generates COMMITMSG.md with a commit message
6. Stops and waits for explicit permission before committing and pushing

/pr-cleanup <ticket-number> handles post-merge housekeeping:

• Checks PR status via gh pr view before any destructive action (learned that one the hard way)
• Removes the worktree and deletes the branch
• Pulls main to get the merged changelog files
• Migrates changelogs to a dedicated KB repository
• Re-indexes the KB from the new location

/pr-resume is for when I come back to a PR after a break. It reads the plan, changelog, git state, and GitHub PR status, then gives me a "here's where you left off" summary.

2. Hooks capture changes without any manual work

Claude Code hooks are shell scripts that fire on lifecycle events. Two of them make the changelog system run on autopilot:

The Stop hook fires after every conversation turn. It reads the conversation transcript (JSONL format), pulls out the last human message and Claude's response, and appends a timestamped entry to CHANGELOG.md:

## 2026-02-26 14:32:11

**I was asked:** How should we handle the edge case where...

**I replied:** The cleanest approach is to...

---

No manual logging. The development conversation just accumulates in CHANGELOG.md as a side effect of working. The hook skips logging when you're on main, verifies the branch matches current-project.txt, and only runs when a changelog folder exists.

The SessionStart hook fires when a session begins, resumes, or recovers from context compaction. It loads PR summary files if they exist, or falls back to PLAN.md plus the last 100 lines of CHANGELOG.md. Everything gets injected into Claude's context window as === CURRENT PR CONTEXT ===.

So Claude Code never starts cold on a PR branch. Even after context compaction (when the conversation gets too long and older messages get compressed), the hook feeds the PR context right back in.

3. Hybrid search indexing

At PR close, the system:

• Generates vector embeddings (text-embedding-3-small, 1536 dimensions)
• Builds PostgreSQL full-text search vectors
• Extracts metadata (project, issue ID, document type)
• Stores everything in Supabase with content hashing for deduplication

Search uses 70% semantic similarity and 30% full-text scoring, with a boost for exact issue ID matches.

4. Querying the KB during development

When I start work on a new feature, Claude Code can search the knowledge base:

"How was video encoding error handling done before?"
-> Returns: PNX-045 PLAN.md, CHANGELOG.md with the full debugging session

"Find past implementations of websocket retry logic"
-> Returns: ranked results with content snippets from relevant PRs

Claude Code sees the current code, but it also sees the history and reasoning behind past changes.

The full pipeline

/pr-start pnx-123-feature apps/website
  |
  |  Creates worktree + branch + changelog folder
  |  Fetches Notion ticket, populates PLAN.md
  |  User opens new session in worktree
  |
  v
[Development]
  |
  |  SessionStart hook: loads PLAN.md + CHANGELOG.md into context
  |  Stop hook: appends "I was asked / I replied" after each turn
  |
  v
/pr-close
  |
  |  Summary, condense, KB ingest, Notion update, commit msg
  |  Waits for permission, then commit + push
  |
  v
[PR merged on GitHub]
  |
  v
/pr-cleanup 123
  |
  |  Verify merge via gh CLI
  |  Remove worktree, delete branch
  |  Pull main, migrate changelogs to KB repo, re-index
  |
  v
[Future development]
  |
  Claude Code queries KB: "How was X done before?"
  Semantic + FTS search returns relevant past PRs
  Plans, conversation logs, code examples inform new work

Architecture

• Storage: Supabase PostgreSQL with pgvector
• Embeddings: text-embedding-3-small via OpenRouter
• Search: Postgres RPC functions (cosine similarity + ts_rank)
• Ingestion: TypeScript CLI with SHA256 change detection
• Access: HTTP API exposed through an MCP bridge to Claude Code
• Tickets: Notion MCP (migrated from Linear UTCP)
• Orchestration: Claude Code slash commands + lifecycle hooks

What I actually get out of this

Reasoning, not just diffs. Git blame tells you what changed. This gives you why it changed, what alternatives came up, and what I learned during the PR.

Documentation I never have to write. The Stop hook logs every conversation turn. The SessionStart hook reloads context on startup. It all happens without me doing anything extra.

Compound returns. 450+ documents means Claude Code has already seen how this codebase handles auth, video encoding, deployment, error handling, and dozens of other things. Each PR adds to that.

No lost context. When Claude Code compacts the conversation mid-session, the SessionStart hook feeds the PR context right back in.

Low friction. /pr-start to begin, work normally, /pr-close to finish. Changelog capture, indexing, and ticket updates happen in between.

What's in this repo

.claude/
  commands/          # Claude Code slash commands
    pr-start.md      # Create worktree + changelog folder + populate from ticket
    pr-close.md      # Summary, condense, index, update ticket, commit
    pr-cleanup.md    # Post-merge: remove worktree, migrate changelogs, re-index
    pr-resume.md     # Rehydrate context when returning to a PR
    pr-summary.md    # Generate code-level PR summary
    pr-condense.md   # Shrink verbose changelog files
    pr-commitmsg.md  # Generate commit message from changes
    log-note.md      # Append timestamped entry to changelog
    log-journal.md   # Log entry to Notion journal (requires Notion MCP)
  hooks/
    on-stop.sh       # Auto-logs each conversation turn to CHANGELOG.md
    on-session-start.sh  # Rehydrates PR context on session start/resume
  skills/
    git-worktrees/   # Worktree management for isolated PR work

apps/code-knowledge-base/   # Core KB package (TypeScript)
  src/
    cli.ts           # CLI: search, list, get, stats, ingest, migrate
    client.ts        # Supabase client and RPC wrappers
    embeddings.ts    # OpenRouter embedding API
    ingest.ts        # Index changelog files into database
    migrate.ts       # Move files to sister KB repo
    config.ts        # Constants, environment helpers
    types.ts         # Shared types
    utils.ts         # Hashing, text extraction, metadata parsing
  supabase/migrations/
    001_*.sql         # Table, indexes, RLS policies
    002_*.sql         # Search RPC functions (hybrid semantic + FTS)
  test/              # Unit and relevance tests

Things you'll want to customize

• Issue prefix: The PNX- pattern is hardcoded in utils.ts, migrate.ts, and the SQL functions. Replace with your own prefix (JIRA-, GH-, PROJ-, etc.)
• Branch prefix: The carlos/ owner prefix in commands and skill docs. Replace with your own name or team prefix.
• Ticket tracker: Commands reference Notion MCP for ticket management. Swap for your own (Linear, Jira, GitHub Issues, etc.)
• Package scope: @your-org/ in package.json. Replace with your npm scope.
• Paths: Worktree and KB repo paths use $HOME/dev/YourProject placeholders. Update to match your directory structure.
• Test fixtures: test/kb-test-queries.json contains queries against the author's KB data. Replace with queries relevant to your own indexed documents.

The takeaway

The AI conversation log turns out to be great documentation. By indexing it for future sessions, every PR becomes searchable institutional knowledge that the next session can build on.

I'm searching my own development history to inform the next change, not docs or Stack Overflow.