systems embeddings and kuzu

Embeddings and Kuzu

Active contributors: Bryan

Embeddings and Kuzu are optional local extensions. src/lode/embeddings.py calls a TEI-compatible embedding endpoint, while src/lode/kuzu_store.py projects SQLite facts into an embedded Kuzu graph database.

Directory layout

src/lode/embeddings.py # HTTP embedding client with retry
src/lode/kuzu_store.py # SQLite-to-Kuzu projection
src/lode/storage.py    # embedding queue, vectors, and source facts
docker-compose.yml     # local TEI service wiring

Key abstractions

Name	File	Description
embed_texts()	src/lode/embeddings.py	Sends text batches to a local embedding endpoint.
pending_embedding_nodes()	src/lode/storage.py	Reads queued nodes for embedding.
upsert_embedding()	src/lode/storage.py	Stores vectors and model metadata.
sync_from_sqlite()	src/lode/kuzu_store.py	Rebuilds the Kuzu projection from SQLite.
reset_schema()	src/lode/kuzu_store.py	Recreates Kuzu node and edge tables.

How embeddings work

replace_file_index() in src/lode/storage.py queues functions, methods, classes, routes, and doc sections that have content hashes. The lode embed command in src/lode/cli.py reads queued nodes, calls embed_texts() in src/lode/embeddings.py, and persists vectors through upsert_embedding().

How Kuzu projection works

lode kuzu-sync calls sync_from_sqlite() in src/lode/kuzu_store.py. The implementation resets Kuzu schema, inserts SQLite nodes into a Node table, and inserts SQLite edges into a LINK relationship table.

graph LR
    SQLite[(SQLite facts)] --> Queue[embedding_queue]
    Queue --> TEI[local TEI endpoint]
    TEI --> Vectors[(embeddings)]
    SQLite --> Kuzu[(Kuzu graph)]

Loading

Integration points

• docker-compose.yml runs ghcr.io/huggingface/text-embeddings-inference:cpu-1.9.
• src/lode/config.py defines SQLite and Kuzu data paths.
• pyproject.toml puts Kuzu behind the kuzu optional dependency.

Entry points for modification

Change embedding request behavior in embed_texts() in src/lode/embeddings.py. Change projected graph schema in reset_schema() and sync_from_sqlite() in src/lode/kuzu_store.py, then verify tests that cover optional dependency behavior.

Key source files

File	Purpose
src/lode/embeddings.py	Local embedding HTTP client.
src/lode/kuzu_store.py	Kuzu projection.
src/lode/storage.py	Embedding queue and vector persistence.
docker-compose.yml	Local embedding service.
pyproject.toml	Optional dependency declaration.

Related pages

• Dependencies
• Configuration
• Search and context