Skip to content

squirrelsoft-dev/infinite-memory

BranchesTags

Repository files navigation

Infinite Memory for Claude Code

Version: 0.1.0-alpha Status: Alpha Release - Ready for Testing License: MIT

Overview

Infinite Memory is a semantic memory system for Claude Code that enables persistent, project-aware context retention through vector embeddings and intelligent retrieval. Built on Pixeltable, it provides fast code indexing, natural language search, and incremental updates without requiring external dependencies.

Key Features

  • Semantic Code Search: Find relevant code using natural language queries
  • Fast Indexing: 1,500+ files per second with incremental updates
  • Local-First: Complete privacy, no external API calls, all data stored locally
  • MCP Integration: Native integration with Claude Code via MCP protocol
  • High Search Quality: 98% relevance rate on diverse query types
  • Incremental Updates: Hash-based change detection for efficient re-indexing

Quick Start

Prerequisites

  • Python 3.11 or higher
  • Claude Code CLI
  • 4GB RAM minimum (8GB recommended)
  • 500MB free disk space + ~2KB per file you'll index

Installation

Option 1: Plugin Installation (Recommended)

Install as a Claude Code plugin for the best experience:

/plugin install infinite-memory@squirrelsoft-marketplace

That's it! The plugin automatically:

  • ✅ Configures the MCP server
  • ✅ Adds /index-project, /search, /status commands
  • ✅ Enables auto-indexing on file save
  • ✅ Activates semantic search Skills

Get Started:

/index-project          # Index your project
/search authentication  # Try a search

See Plugin Guide for complete documentation.

Option 2: Manual Installation

For development or customization, install manually:

# Clone the repository
git clone https://github.com/squirrelsoft-dev/infinite-memory.git
cd infinite-memory

# Create virtual environment and install dependencies
uv venv
uv pip install -e ".[dev]"

# Verify installation
.venv/bin/pytest tests/ -v --tb=short

Expected output: 403 passed, 2 skipped

Then configure MCP manually (see below).

Configure MCP for Claude Code (Manual Installation Only)

Add Infinite Memory to your Claude Code MCP configuration:

{
  "mcpServers": {
    "infinite-memory": {
      "command": "python",
      "args": [
        "-m",
        "infinite_memory.server"
      ],
      "cwd": "/path/to/infinite-memory",
      "env": {
        "PYTHONPATH": "/path/to/infinite-memory/src"
      }
    }
  }
}

Replace /path/to/infinite-memory with the absolute path to your cloned repository.

First Use

  1. Start Claude Code with MCP enabled

  2. Index your first project:

    Can you index my project at /path/to/my/project?

    Claude Code will use the index_project tool to scan and index your code.

  3. Run your first search:

    Search my memory for "authentication and session management"

    Claude Code will use the query_memory tool to find relevant code.

  4. Update a file:

    Index the file /path/to/my/project/auth.py

    Claude Code will use the index_file tool for incremental updates.

Example Queries

Conceptual Queries:

  • "How does database connection pooling work?"
  • "Show me error handling patterns"
  • "Where is logging configured?"

Pattern Queries:

  • "Find all class definitions with inheritance"
  • "Show me decorator usage"
  • "Where are context managers used?"

Specific Queries:

  • "Find the main application entry point"
  • "Show me the configuration file"
  • "Where is the User model defined?"

Cross-File Queries:

  • "Where is the authenticate function called?"
  • "Show me all files using the CacheManager"
  • "Find tests for the API endpoint"

Performance

Based on comprehensive benchmarking on realistic codebases:

Metric Target Actual Status
Indexing Throughput <10 min for 500 files 1,500+ files/sec ✅ 1,800x faster
Search Quality >80% relevance 98% relevance ✅ Exceeds target
Search Latency <200ms (10K files) <200ms ✅ Meets target
Storage Overhead ~2KB per file ~2KB per file ✅ As designed
Memory Usage <500MB ~400MB ✅ Efficient

Real-World Performance

100 Files (Small Project):

  • Indexing: ~0.07 seconds
  • Search: <100ms
  • Storage: ~200KB

500 Files (Medium Project):

  • Indexing: ~0.3 seconds
  • Search: <150ms
  • Storage: ~1MB

1,000 Files (Large Project):

  • Indexing: ~0.6 seconds
  • Search: <200ms
  • Storage: ~2MB

Architecture

High-Level Design

┌─────────────────────────────────────────────────────────┐
│                    Claude Code CLI                       │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ User Commands│  │  MCP Client  │  │ Search Tools │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
└─────────┼──────────────────┼──────────────────┼──────────┘
          │                  │                  │
          └──────────────────┴──────────────────┘
                             │
                    MCP Protocol (stdio)
                             │
          ┌──────────────────┴──────────────────┐
          │      Python MCP Server               │
          │  ┌────────────────────────────────┐  │
          │  │  MCP Tools                     │  │
          │  │  • index_project               │  │
          │  │  • query_memory                │  │
          │  │  • index_file                  │  │
          │  └────────────────────────────────┘  │
          │  ┌────────────────────────────────┐  │
          │  │  Core Modules                  │  │
          │  │  • Indexer  • Search           │  │
          │  │  • Config   • Utilities        │  │
          │  └────────────────────────────────┘  │
          └──────────────┬──────────────────────┘
                         │
          ┌──────────────┴──────────────────────┐
          │         Pixeltable Database          │
          │  ┌────────────────────────────────┐  │
          │  │  Tables                        │  │
          │  │  • code_files (with embeddings)│  │
          │  └────────────────────────────────┘  │
          │  ┌────────────────────────────────┐  │
          │  │  Indexes                       │  │
          │  │  • HNSW Embedding Index        │  │
          │  └────────────────────────────────┘  │
          └─────────────────────────────────────┘

Technology Stack

  • Runtime: Python 3.11+
  • Database: Pixeltable 0.5+ (vector database with built-in embeddings)
  • Embeddings: Sentence Transformers (all-MiniLM-L6-v2 model, 384 dimensions)
  • Protocol: MCP (Model Context Protocol)
  • Transport: stdio (standard input/output)

Key Components

MCP Server (src/infinite_memory/server.py):

  • Entry point for the MCP server
  • Exposes 3 tools: index_project, query_memory, index_file
  • Handles validation, error handling, and logging

Indexer Module (src/infinite_memory/indexer/):

  • File discovery with gitignore support
  • Content extraction from supported file types
  • Hash-based change detection
  • Batch processing with progress tracking

Search Module (src/infinite_memory/search/):

  • Semantic search using vector similarity
  • Result formatting with tiered content loading
  • Context extraction with keyword highlighting

Database Module (src/infinite_memory/indexer/database.py):

  • Pixeltable table management
  • Embedding index creation
  • CRUD operations on code files

Supported File Types

Currently supported file extensions:

  • Python: .py
  • JavaScript/TypeScript: .js, .ts
  • Documentation: .md, .txt
  • Configuration: .json, .yaml, .yml

Additional file types can be added in future releases.

Usage

MCP Tools

All interactions happen through Claude Code using the 3 MCP tools. See API Documentation for detailed tool specifications.

1. Index a Project

Index my project at /Users/me/projects/myapp

Returns statistics about files processed, indexed, and any errors.

2. Query the Memory

Search for "authentication middleware"

Returns relevant code snippets with similarity scores and file paths.

3. Index a Single File

Index the file /Users/me/projects/myapp/auth.py

Incrementally updates a single file in the database.

Advanced Usage

Force Re-indexing:

Re-index /Users/me/projects/myapp even if files haven't changed

Adjust Search Parameters:

Search for "logging setup" with max 10 results and minimum similarity 0.8

Check Indexing Status:

How many files are indexed in my project?

Troubleshooting

Common Issues

Issue: MCP server won't start

Solution:

  1. Verify Python version: python --version (must be 3.11+)
  2. Check installation: uv pip list | grep infinite-memory
  3. Review MCP configuration path in Claude Code settings
  4. Check server logs for specific errors

Issue: Indexing fails with "permission denied"

Solution:

  • Ensure you have read permissions on the project directory
  • Avoid indexing system directories like /etc, /sys, /proc, /dev
  • Check that project path is absolute (not relative)

Issue: Search returns no results

Solution:

  1. Verify project was indexed: Check returned statistics from index_project
  2. Try broader queries first: "database" before "connection pooling optimization"
  3. Check similarity threshold isn't too high (default is 0.7)
  4. Ensure indexed files contain relevant content

Issue: Slow performance

Solution:

  • Close other memory-intensive applications
  • Ensure at least 4GB RAM available
  • Check if using HDD instead of SSD (impacts Pixeltable performance)
  • Monitor system resources during indexing

Issue: File not updating after changes

Solution:

  • Use force_update=True parameter when re-indexing
  • Verify file hash changed (file was actually modified)
  • Check file permissions (must be readable)

Debug Mode

Enable detailed logging by setting environment variable:

export ENV=development
python -m infinite_memory.server

This includes tracebacks in error responses and verbose logging.

Getting Help

Alpha Release Limitations

This is an alpha release for early testing and feedback. Known limitations:

Not Yet Implemented (Planned for Beta)

  • Auto-indexing hooks: Automatic indexing on file save (currently manual)
  • Multi-project support: Switching between multiple indexed projects
  • Search filters: Filter by file type, date, path pattern
  • Custom exclude patterns: Currently only respects .gitignore
  • Query history: Save and replay previous queries
  • Result ranking customization: Boost recent/popular files

Known Issues

  1. Generic pattern queries: Very generic queries like "find all classes" may return lower quality results. Use more specific language.

  2. Model loading overhead: First indexing operation loads ~200MB embedding model (one-time per session).

  3. Windows compatibility: System is untested on Windows. macOS and Linux are supported.

  4. Large file handling: Files >1MB are indexed but may impact performance. Consider excluding large files.

  5. Binary files: Binary files are skipped during indexing (expected behavior).

Acceptable for Alpha

  • Single-user only (no concurrent database access)
  • Local storage only (no cloud sync)
  • Limited language support (see supported file types)
  • No incremental search (must re-index on schema changes)
  • Basic error messages (improved error reporting in beta)

Development

Setup for Development

# Clone and install with dev dependencies
git clone https://github.com/your-org/infinite-memory.git
cd infinite-memory
uv venv
uv pip install -e ".[dev]"

# Run tests
.venv/bin/pytest tests/ -v

# Run with coverage
.venv/bin/pytest tests/ -v --cov=src --cov-report=term-missing

# Format code
uv run black .

# Lint
uv run ruff check . --fix

Running Tests

# All tests
.venv/bin/pytest tests/ -v

# Specific test file
.venv/bin/pytest tests/test_embedding_index.py -v

# Performance benchmarks
.venv/bin/pytest tests/benchmark_indexing_performance.py -v

# Search quality validation
.venv/bin/pytest tests/test_search_quality_validation.py -v

Project Structure

infinite-memory/
├── src/
│   └── infinite_memory/          # Main Python package
│       ├── __init__.py
│       ├── server.py             # MCP server entry point
│       ├── exceptions.py         # Custom exceptions
│       ├── indexer/              # Indexing module
│       │   ├── batch.py          # Batch processing
│       │   ├── database.py       # Database operations
│       │   ├── discovery.py      # File discovery
│       │   ├── errors.py         # Error handling
│       │   └── incremental.py    # Incremental updates
│       ├── search/               # Search module
│       │   ├── context.py        # Context extraction
│       │   ├── formatters.py     # Result formatting
│       │   └── searcher.py       # Semantic search
│       ├── config/               # Configuration management
│       │   └── project.py        # Project config
│       └── utils/                # Shared utilities
│           └── file_utils.py
├── tests/                        # Test suite (403 tests)
│   ├── test_*.py                 # Unit tests
│   ├── benchmark_*.py            # Performance benchmarks
│   └── sample_files/             # Sample code for testing
├── docs/                         # Documentation
│   ├── API.md                    # API reference
│   ├── alpha-user-test-guide.md  # Testing guide
│   └── epics/                    # Epic tracking
├── pyproject.toml                # Python project config
├── uv.lock                       # Dependency lock file
├── CLAUDE.md                     # Developer guide
└── README.md                     # This file

Contributing

This project is in alpha testing. We welcome bug reports, feedback, and feature suggestions.

Reporting Issues

When reporting issues, please include:

  1. Python version and OS
  2. Steps to reproduce
  3. Expected vs actual behavior
  4. Error messages and logs
  5. Sample code or project structure (if applicable)

Feedback

We're particularly interested in feedback on:

  • Search quality on your codebase
  • Performance on different project sizes
  • Usability of MCP tools through Claude Code
  • Missing features that would be valuable

Documentation

License

MIT License - See LICENSE file for details.

Acknowledgments

Built with:

Roadmap

Beta Release (Planned)

  • Auto-indexing hooks for file changes
  • Multi-project workspace support
  • Search filters and facets
  • Custom exclude patterns
  • Query history and saved searches
  • Improved error messages
  • Windows support

v1.0 Release (Future)

  • Advanced search features (hybrid search, reranking)
  • Performance monitoring and telemetry
  • Plugin system for custom file types
  • Search result explanations
  • API for programmatic access
  • Cloud sync options (optional)

Contact

Status: Alpha Release - Ready for Testing Last Updated: 2025-11-28 Version: 0.1.0-alpha

About

No description, website, or topics provided.

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages