Lexora Knowledge

Lexora Knowledge is an enterprise-grade legal knowledge engineering platform that transforms raw legal documents into high-quality, structured Knowledge Graph assets optimized for legal AI and search applications.

Built using Domain-Driven Design (DDD) principles and structured parsing workflows, it acts as a data compiler that ingests legal raw structures and outputs standardized, versioned Knowledge Releases.

---

🏗️ Architecture & Folder Structure

The project strictly follows a decoupled DDD layout designed for multi-year extensibility:

├── domain/                  # DDD Business Objects (Chapter, Article, Clause, Relationship, etc.)
├── sources/                 # Pure raw data crawlers (VBPL BFS crawler)
├── parsers/                 # HTML physical structure parsers (HTML -> AST -> Node Hierarchy)
├── metadata/                # Metadata extractors (Effective dates, Signers, Legal Bases)
├── relationships/           # Citation & cross-reference link extractors
├── datasets/                # Release Exporter (Serializes to JSONL releases - Single Source of Truth)
├── graph/                   # Graph DB loaders and providers (Neo4j importer)
├── infrastructure/          # Client drivers, Pydantic-settings configurations
├── pipelines/               # Orchestrated E2E pipeline runners
├── tests/                   # Isolated Unit and Integration tests
└── docker-compose.yml       # Local infrastructure dependencies

---

🛠️ Technology Stack

• Runtime: Python >= 3.10
• Database: Neo4j Graph Database
• Configuration: Pydantic Settings (type-validated env variables)
• Formatting & Linting: Ruff
• Testing: Pytest

---

🚀 Quickstart & Setup

1. Setup Virtual Environment

This project runs inside a virtual environment. You can use standard virtualenv or Fish Shell's VirtualFish:

vf new lexora-knowledge
vf activate lexora-knowledge
pip install -r requirements.txt

2. Configure Environment

Copy the example environment settings and customize the credentials if necessary:

cp .env.example .env

3. Spin Up Infrastructure (Neo4j)

Launch the local Neo4j Docker container:

make infra-up

(Wait 10 seconds for the database engine to initialize and listen on Bolt port 7687 before running imports).

---

🏃 Run Orchestrated Pipelines

The platform compiles data through distinct pipeline execution stages:

Stage	Command	Description
1. Crawl	make run-crawl	BFS crawl seeds to download raw payloads into data/raw/
2. Ingest	make run-ingest	Cleans HTML, parses hierarchy, extracts metadata & citations, and writes a versioned Knowledge Release to data/releases/v1.0/
3. Import	make run-import	Loads the versioned release files into the active Neo4j database using optimized transactional batches
E2E Run	make E2E	Sequentially executes Crawl, Ingestion, and Graph DB Import

---

📂 Versioned Knowledge Releases (Single Source of Truth)

The output of the Ingestion Pipeline is a packaged folder under data/releases/{version}/ containing:

• nodes.jsonl: Standardized node documents containing metadata properties.
• edges.jsonl: Inter-node structural relationships (e.g. HAS_ARTICLE) and external cross-reference citations (e.g. CITES) with source/target labels.
• statistics.json: Complete count profile of nodes and edges.
• manifest.json: Release metadata tracking version, timestamp, files, and crawled seeds.

Downstream databases (like Neo4j) consume exclusively from these releases to guarantee consistency and idempotency.

---

🧪 Developer Commands

Verify code quality, formatting, and tests using the unified Makefile:

• Run Tests: make test
• Run Linter: make lint
• Run Formatter: make format
• Stop Infrastructure: make infra-down
• Clean Cache & Data: make clean