Lease AI - Multi-Document GraphRAG Chat

AI-powered chat agent for analyzing legal lease documents using Google ADK with FalkorDB Graph + ChromaDB Vector hybrid search.

Quick Start

# 1. Start FalkorDB
docker run -d -p 6379:6379 -p 3000:3000 --name falkordb falkordb/falkordb

# 2. Setup environment (Conda recommended)
conda activate lease
pip install -r requirements.txt

# 3. Set API key
echo "OPENAI_API_KEY=your-key" > .env

# 4. Run servers (Environment-safe)
bash run_server.sh 9231   # Terminal 1: FastAPI backend
bash run_adk.sh           # Terminal 2: ADK chat interface

[!IMPORTANT] > Avoid python3 server.py if you encounter opentelemetry ImportErrors. This usually happens when the system Python (e.g., 3.13) collides with the conda environment. Always use python with the environment activated or the provided .sh helper scripts.

---

API Endpoints

Endpoint	Method	Description
/upload	POST	Upload PDF → returns doc_id
/documents	GET	List all indexed documents
/documents/{doc_id}	DELETE	Delete document from all stores
/chat	POST	Q&A with doc_id routing
/extract	POST	Extract structured summary
/evaluate	POST	Run automated quality tests

---

Lease Summary Template

The /extract endpoint extracts structured data based on this template:

Field	Type	Description
Parties
Landlord	Text	Property owner name
Tenant	Text	Lessee name
Premises
Address	Text	Property address
Size	Number	Square footage
Key Dates
Lease Date	Date	Execution date
Commencement	Date	Start date
Expiration	Date	End date
Financial
Monthly Rent	Number	Base rent amount
Security Deposit	Number	Deposit amount
Options
Renewal Options	Number	Count of renewal options
Notice Period	Text	Early/late notice requirements

Example Extract Output

{
  "parties_and_premises": {
    "landlord": { "value": "Sell Family Partners", "source": "Page 2" },
    "tenant": { "value": "Nelly's Italian Cafe", "source": "Page 2" },
    "address": {
      "value": "Spring Hill Shopping Center, TN",
      "source": "Page 2"
    },
    "size_sqft": { "value": 1200, "source": "Page 2" }
  },
  "key_dates": {
    "lease_date": { "value": "2014-01-17", "source": "Page 2" },
    "commencement_date": { "value": "2014-04-01", "source": "Page 2" }
  },
  "financial": {
    "security_deposit": { "value": 1900, "source": "Page 2" },
    "monthly_rent": { "value": "See Exhibit C", "source": "Page 2" }
  }
}

---

Source Citation Format

Every chat response includes page citations for traceability.

Example Chat Response

{
  "answer": "The security deposit for the lease is $1,900.",
  "relevant_sections": [
    "SECTION 1.1 BASIC LEASE PROVISIONS",
    "ARTICLE 40. SECURITY DEPOSIT"
  ],
  "page_citations": [2, 30],
  "confidence": "high"
}

Confidence Levels:

• high: Answer found directly in retrieved context
• medium: Answer inferred from related context
• low: Limited relevant context found

---

Document Preprocessing

When a PDF is uploaded, it goes through this pipeline:

PDF File
   ↓
┌─ PyMuPDF Extraction ─────────────────────┐
│  - Extract text per page                 │
│  - Preserve page numbers                 │
│  - Handle multi-column layouts           │
└──────────────────────────────────────────┘
   ↓
┌─ Page-Level Chunking ────────────────────┐
│  - Each page = one chunk                 │
│  - Maintains document structure          │
│  - Preserves legal clause boundaries     │
└──────────────────────────────────────────┘
   ↓
┌─ Dual Indexing ──────────────────────────┐
│  - FalkorDB: Graph with relationships    │
│  - ChromaDB: Vector embeddings           │
└──────────────────────────────────────────┘

---

Clause Graph Structure

Each document creates a separate graph in FalkorDB:

Graph: lease_{doc_id}

┌─────────────┐
│  Document   │ name, total_pages
└──────┬──────┘
       │ HAS_CLAUSE
       ↓
┌──────────────┐    NEXT    ┌──────────────┐    NEXT    ┌──────────────┐
│  Clause P1   │ ────────→  │  Clause P2   │ ────────→  │  Clause P3   │
└──────────────┘            └──────────────┘            └──────────────┘
       ↑                           ↑                           ↑
       │ MENTIONED_IN              │ MENTIONED_IN              │
       │                           │                           │
┌──────────────┐            ┌──────────────┐            ┌──────────────┐
│   Landlord   │            │    Tenant    │            │     Rent     │
│   Entity     │            │    Entity    │            │    Entity    │
└──────────────┘            └──────────────┘            └──────────────┘

Node Types:

• Document: Root node with metadata
• Clause: One per page, contains full text
• Entity: Extracted entities (Landlord, Tenant, Rent, etc.)

Relationship Types:

• HAS_CLAUSE: Document → Clause
• NEXT: Sequential reading order between clauses
• MENTIONED_IN: Entity → Clause where it appears

---

Multi-Document Routing

Each uploaded document is completely isolated:

Upload Doc A → doc_id: "abc123"
┌─ FalkorDB: lease_abc123 ─────────────────┐
│  Separate graph with A's clauses         │
└──────────────────────────────────────────┘
┌─ ChromaDB: lease_abc123 ─────────────────┐
│  Separate collection with A's embeddings │
└──────────────────────────────────────────┘

Upload Doc B → doc_id: "xyz789"
┌─ FalkorDB: lease_xyz789 ─────────────────┐
│  Separate graph with B's clauses         │
└──────────────────────────────────────────┘
┌─ ChromaDB: lease_xyz789 ─────────────────┐
│  Separate collection with B's embeddings │
└──────────────────────────────────────────┘

Chat Routing:

{"doc_id": "abc123", "question": "..."}  → Queries only abc123's stores
{"doc_id": "xyz789", "question": "..."}  → Queries only xyz789's stores

---

Evaluation Criteria

The /evaluate endpoint runs automated quality tests:

Test Generation

1. Sample random clauses from the document's graph
2. Use LLM to generate Q&A pairs from clause text
3. Each test has: Question, Expected Answer, Reference Page

Grading Metrics

Metric	Type	Description
Citation Accuracy	Deterministic	Was the reference page cited in response?
Answer Faithfulness	LLM-as-Judge	Does answer match expected (1-5 scale)?

Pass Criteria: Citation correct AND faithfulness ≥ 3

Example Evaluation Response

{
  "doc_id": "abc123",
  "health_score": 67,
  "tests_passed": 2,
  "tests_total": 3,
  "results": [
    {
      "question": "Who is the broker for this lease?",
      "expected_answer": "Turner & Associates Realty, Inc.",
      "generated_answer": "Turner & Associates Realty, Inc. (Page 18)",
      "reference_page": 18,
      "cited_pages": [18],
      "citation_pass": true,
      "faithfulness_score": 5,
      "status": "pass"
    }
  ]
}

---

Project Structure

lease-ai/
├── lease_assistant/         # ADK agent package
│   └── agent.py             # Tools: upload, chat, evaluate, delete
├── server.py                # FastAPI backend
├── src/
│   ├── graph_store.py       # FalkorDB per-doc graphs
│   ├── vector_store.py      # ChromaDB per-doc collections
│   ├── chat.py              # Hybrid Vector+Graph retrieval
│   ├── evaluator.py         # Automated quality testing
│   ├── extractor.py         # Structured data extraction
│   └── models.py            # Pydantic schemas
├── documents/               # Uploaded PDFs
└── output/                  # Vector store + summaries

Requirements

• Python 3.11+
• FalkorDB (Docker)
• OpenAI API key
• Conda environment recommended