Legal RAG System with Hy

A jurisdiction-aware Retrieval-Augmented Generation (RAG) system for legal research, built with Hy and org-mode literate programming.

Overview

The Legal RAG System provides an efficient, accurate, and jurisdiction-aware approach to legal research by:

• Retrieving relevant legal documents from a vector database based on queries
• Enhancing document relevance with jurisdiction-aware scoring
• Generating accurate, well-cited answers using LLMs
• Providing an efficient embedding cache with TTL to reduce API costs
• Supporting multiple document types (cases, statutes, regulations)

Features

• Jurisdiction-Aware Retrieval: Boosts relevance based on jurisdictional hierarchy
• Citation Network Analysis: Weights documents by citation relationships
• Embedding Cache: TTL-based caching with version tracking
• Failure Handling: Graceful degradation with fallbacks
• Evaluation Framework: Comprehensive metrics for RAG performance
• Optimized Vector Search: Efficient similarity search with FAISS

Installation

# Clone the repository
git clone https://github.com/defrecord/legal-rag-hy.git
cd legal-rag-hy

# Ensure you have Hy 1.0.0 installed
pip install "hy==1.0.0"

# Install dependencies (option 1 - with pip)
pip install -e .

# OR with Poetry (recommended)
poetry install

# Set your OpenAI API key
export OPENAI_API_KEY=your_api_key_here

Quick Start for Conference Demo

For the conference demo, use our simplified setup:

# Install core dependencies
pip install "hy==1.0.0" openai faiss-cpu numpy

# Run the simple demo
cd legal-rag-hy
./demo-simple.hy

# Run the full demo (requires OpenAI API key)
cd legal-rag-hy
python3 -m pip install -e .
hy src/legal_rag/demo.hy

Usage

Basic usage example:

;; Hy 1.0 compatible code
(import [legal_rag.system [LegalRAGSystem]])

;; Initialize the system
(setv rag-system (LegalRAGSystem 
                   :embedding-model "text-embedding-3-large"
                   :llm-model "gpt-4o"
                   :jurisdiction "CA-9"))

;; Query the system
(setv query "What constitutes fair use under copyright law?")
(setv response (. rag-system (query query)))

;; Display the response with citations
(print response.answer)
(print "\nCitations:")
(for [citation response.citations]
  (print (get citation "citation")))

Important: Note the Hy 1.0 method call syntax: `(. object (method args))` instead of `object.method(args)`.

Architecture

The system is built with a modular architecture:

1. Query Processing: Pre-processes legal queries for optimal retrieval
2. Vector Search: Jurisdiction-aware document retrieval with FAISS
3. Context Formation: Structures retrieved documents for the LLM
4. Answer Generation: Produces accurate, well-cited responses
5. Embedding Cache: Optimizes performance and reduces API costs
6. Evaluation Module: Assesses system performance with multiple metrics

Documentation

Comprehensive documentation with full literate programming support:

• Installation Guide
• Usage Guide
• System Architecture
• Evaluation Framework
• Jurisdiction Handling
• Production Deployment Guide

Generate HTML documentation:

make docs

Examples

Interactive examples with org-mode tangle/detangle support:

• Legal RAG Demo Notebook - Demonstrates the system with visualizations

Run the examples:

make notebook

Contributing

Contributions are welcome! Please read our Contributing Guide for details on our code of conduct, the process for submitting pull requests, and our literate programming workflow.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributors

• Kushagra Kumar - Lead Developer, ML Systems
• Jason Walsh - Project Lead, System Architecture
• Xianglong Tao - Developer, Testing Infrastructure
• Aidan Pace - Developer, Infrastructure Integration
• Sean Jensen-Grey - Research Advisor

Acknowledgments

• The Hy community for the excellent Lisp dialect
• FAISS developers for the efficient similarity search
• LangChain contributors for inspiration
• DefRecord team for domain expertise

Citation

If you use this system in your research, please cite:

Kumar, K., Walsh, J., Tao, X., & Pace, A. (2025). Legal RAG: A Jurisdiction-Aware Retrieval-Augmented Generation System for Legal Research. In Proceedings of the Legal Informatics and Technology Conference (LITCon 2025). New York, NY.

BibTeX:

@inproceedings{LegalRAG2025,
  author       = {Kumar, Kushagra and Walsh, Jason and Tao, Xianglong and Pace, Aidan},
  title        = {Legal {RAG}: {A} Jurisdiction-Aware Retrieval-Augmented Generation System for Legal Research},
  booktitle    = {Proceedings of the Legal Informatics and Technology Conference},
  series       = {LITCon 2025},
  year         = {2025},
  location     = {New York, NY},
  publisher    = {ACM},
  doi          = {10.1145/3579095.3579132}
}