Skip to content
BranchesTags

Repository files navigation

GraphRAG: Advanced Graph-Based Retrieval Augmented Generation

Python Version License Version CI

Transform mindmaps and documents into intelligent knowledge graphs with AI-powered querying.

GraphRAG converts structured content (mindmaps, documents, research papers) into interconnected knowledge graphs, enabling sophisticated semantic search and intelligent question-answering using advanced retrieval-augmented generation techniques.

✨ Key Features

  • 🤖 Multi-Format Parsing: JSON, OPML, Markdown, and custom formats
  • 🕸️ Dual Graph Backends: NetworkX (development) + Neo4j (production-scale)
  • 🧠 Intelligent RAG: Entity extraction + graph traversal + LLM synthesis
  • 🔄 Multiple LLM Providers: OpenAI, Anthropic Claude, OpenRouter
  • 🎯 Vector Search: Voyage AI embeddings for semantic similarity
  • 🖥️ Rich CLI: Interactive terminal interface with progress indicators
  • 🔒 Production Ready: Comprehensive error handling and validation
  • 🧪 Fully Tested: Complete test suite with CI/CD pipeline

🚀 Quick Start

Installation

# Install from source
git clone https://github.com/yourusername/graphrag.git
cd graphrag
pip install -e ".[dev]"

# Or install from PyPI (once published)
pip install graphrag

Basic Usage

# Load sample knowledge base
graphrag load-sample

# Ask intelligent questions
graphrag query "What are the key principles of injury prevention?"

# Search for concepts
graphrag search "recovery strategies"

# Explore your knowledge graph
graphrag stats

📋 Features

✅ Implemented

  • Multi-format Mindmap Parsing: JSON, OPML, and Markdown outlines
  • Dual Backend Support: NetworkX (local) and Neo4j (production)
  • Intelligent RAG: Entity extraction + graph traversal + LLM synthesis
  • Multiple LLM Providers: OpenAI, Anthropic Claude, and OpenRouter routing
  • Vector Embeddings: Voyage AI integration for hybrid search
  • Comprehensive CLI: Full command-line interface for all operations
  • Error Handling: Robust error handling and validation
  • Testing Framework: Complete test suite with integration tests

🔄 Architecture

Mindmap (JSON/OPML/MD) → Parser → Triples → Graph Backend → RAG Orchestrator → Answer

🎯 Use Cases

Perfect for:

  • Plaud AI Summary Analysis: Convert voice summaries into queryable knowledge
  • Research Synthesis: Build knowledge graphs from literature reviews
  • Meeting Notes: Transform discussion mindmaps into searchable databases
  • Learning Management: Create interactive knowledge bases from course outlines

📖 Detailed Usage

📖 Usage Examples

1. Loading Content

From Various Sources

# Load JSON mindmap
graphrag load my_summary.json --format json

# Load OPML outline
graphrag load my_outline.opml --format opml

# Load Markdown document
graphrag load research_notes.md --format md

# Load sample data to get started
graphrag load-sample

Create Sample Content

# Generate sample files for testing
graphrag create-sample sample.json

2. Intelligent Querying

Basic Questions

# Ask natural language questions
graphrag query "How does acute:chronic workload ratio prevent injuries?"

# Show reasoning and context
graphrag query "What are evidence-based recovery strategies?" --show-context

# Include source citations
graphrag query "Sleep optimization techniques" --show-sources

Advanced Search

# Semantic concept search
graphrag search "periodization principles"

# View knowledge graph statistics
graphrag stats

# Interactive mode for exploration
graphrag interactive

3. Graph Management

# Clear current knowledge base
graphrag clear

# Export graph data
graphrag export graph.json

# Import external knowledge
graphrag import external_data.json

⚙️ Configuration

Environment Setup

Create a .env file in your project root (copy from env-example.txt):

cp env-example.txt .env
# Edit .env with your API keys

Key Configuration Options

# ==========================================
# LLM Configuration (Required for RAG queries)
# ==========================================
OPENROUTER_API_KEY=your_openrouter_api_key
OPENROUTER_MODEL=anthropic/claude-3.5-sonnet

# ==========================================
# Embedding Configuration (Optional)
# ==========================================
VOYAGE_API_KEY=your_voyage_api_key
VOYAGE_MODEL=voyage-large-2-instruct

# ==========================================
# Database Configuration
# ==========================================
USE_LOCAL_GRAPH=true  # NetworkX for development
NEO4J_URI=bolt://localhost:7687  # For Neo4j production
NEO4J_USER=neo4j
NEO4J_PASSWORD=password

# ==========================================
# Application Settings
# ==========================================
LOG_LEVEL=INFO
MAX_CONTEXT_NODES=150
MAX_EVIDENCE_NODES=50
GRAPH_DEPTH=2

🏗️ Architecture Details

Mindmap Parser

  • JSON: Direct parsing with nested structure support
  • OPML: XML parsing with outline hierarchy
  • Markdown: Regex-based heading and list parsing

Graph Schema

Node Types:
├── Concept (main topics, subtopics)
├── Person (if tagged)
├── Organization (if tagged)
└── Evidence (URLs, sources)

Relationships:
├── SUBTOPIC_OF (hierarchy)
├── RELATES_TO (cross-links)
├── SUPPORTED_BY (evidence links)
├── CAUSES/PART_OF (causality/composition)
└── CONTRADICTED_BY (conflicting info)

RAG Pipeline

  1. Entity Extraction: LLM identifies key concepts in question
  2. Graph Search: Fulltext search + neighborhood expansion
  3. Context Building: Synthesize relevant information
  4. Answer Generation: LLM produces final response with citations

🔧 Advanced Configuration

Neo4j Setup (Production)

# Install Neo4j Desktop or Server
# Create fulltext index for search:
CREATE FULLTEXT INDEX conceptTitleIdx FOR (n:Concept) ON EACH [n.title, n.summary]

Custom Mindmap Formats

Extend MindmapParser class for new formats:

class CustomParser(MindmapParser):
    @staticmethod
    def parse_custom_format(file_path: Path) -> MindmapNode:
        # Your parsing logic here
        pass

Custom LLM Providers

Implement LLMProvider interface:

class CustomLLM(LLMProvider):
    async def generate(self, prompt: str, **kwargs) -> str:
        # Your LLM logic here
        pass

🧪 Testing

# Run all tests
python -m pytest test_graphrag.py -v

# Run integration tests
python -m pytest test_graphrag.py::TestIntegration -v

# Run basic validation
python test_graphrag.py

📊 Performance Considerations

Graph Size Limits

  • NetworkX: Good for < 10K nodes (fits in memory)
  • Neo4j: Scales to millions of nodes
  • Context Window: Limit to 150 nodes per query for LLM efficiency

Optimization Tips

  • Use fulltext indexes for concept search
  • Pre-compute embeddings for large graphs
  • Batch operations for bulk loading
  • Implement caching for frequent queries

🔍 Comparison with ChatGPT Approach

✅ Improvements Made

  • Error Handling: Comprehensive validation and graceful failures
  • Multiple Backends: Both local (NetworkX) and production (Neo4j)
  • Provider Flexibility: Support for multiple LLM providers
  • Testing: Full test coverage with integration tests
  • CLI Interface: User-friendly command-line tools
  • Configuration: Environment-based settings management
  • Documentation: Detailed usage examples and architecture docs

🎯 Key Differences

  • Production Ready: Added proper error handling, logging, configuration
  • Extensible: Clean interfaces for adding new parsers/providers
  • User Experience: CLI tools + comprehensive documentation
  • Testing: Automated test suite for reliability
  • Performance: Optimized for both small and large graphs

🚦 Troubleshooting

Common Issues

"No module named 'neo4j'"

pip install neo4j

"API key not found"

# Set environment variables or use .env file
export OPENROUTER_API_KEY=your_key

"Graph query timeout"

# Reduce graph depth in config
echo "GRAPH_DEPTH=1" >> .env

"Memory error with large mindmaps"

# Use Neo4j backend instead of NetworkX
echo "USE_LOCAL_GRAPH=false" >> .env

🛠️ Development

Prerequisites

  • Python 3.8+
  • Git
  • pip (latest version recommended)

Setup for Development

# Clone repository
git clone https://github.com/yourusername/graphrag.git
cd graphrag

# Install in development mode with all dependencies
pip install -e ".[dev]"

# Install pre-commit hooks
pre-commit install

# Run tests
pytest

# Format code
black src/ tests/
isort src/ tests/

Code Quality

This project uses several tools to maintain code quality:

  • Black: Code formatting
  • isort: Import sorting
  • ruff: Fast Python linter
  • mypy: Static type checking
  • pytest: Testing framework
# Run all quality checks
pre-commit run --all-files

# Run specific checks
black --check src/
mypy src/
pytest --cov=graphrag

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Quick Contribution Steps

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/your-feature
  3. Make your changes
  4. Add tests for new functionality
  5. Ensure all tests pass: pytest
  6. Format code: black src/ tests/
  7. Submit a pull request

📚 API Reference

Core Classes

Class Purpose
MindmapParser Parse various mindmap formats (JSON, OPML, Markdown)
GraphBuilder Convert parsed content to graph triples and relationships
RAGOrchestrator Handle RAG queries with entity extraction and LLM synthesis
NetworkXBackend Local graph storage using NetworkX
Neo4jBackend Production graph database using Neo4j

CLI Commands

Command Description
graphrag load <file> Import mindmap/document files
graphrag load-sample Load example knowledge base
graphrag query <question> Ask questions with RAG
graphrag search <term> Find concepts by semantic search
graphrag stats Display knowledge graph statistics
graphrag clear Reset knowledge graph
graphrag interactive Start interactive exploration mode

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • NetworkX: For the excellent graph library
  • Neo4j: For the powerful graph database
  • OpenAI, Anthropic: For LLM capabilities
  • Voyage AI: For vector embeddings
  • Typer & Rich: For the beautiful CLI experience

📞 Support

Transform your documents into intelligent knowledge with GraphRAG
Made with ❤️ for the AI community

🎯 Quick Demo

# 🚀 Get started in minutes
graphrag load-sample

# 🤔 Ask intelligent questions
graphrag query "What are the key principles of injury prevention?"
graphrag query "How does acute:chronic ratio work?" --show-context

# 🔍 Explore your knowledge
graphrag search "recovery strategies"
graphrag stats

# 🎮 Interactive exploration
graphrag interactive

Transform documents into intelligent knowledge bases! 🎯

Built with ❤️ for researchers, developers, and knowledge workers who want to make their documents smarter.

About

No description, website, or topics provided.

Resources

Readme
MIT license

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages