Home

cognee-backfill technical wiki

Implementation and operations reference for the distill-first AI history pipeline.

Repository · Quick start · Issues

---

Agent-memory systems capture what happens after installation, but do not ship a practical historical backfill path. cognee-backfill supplies that missing layer: it turns existing coding-assistant history into a compact, searchable Cognee knowledge graph.

The pipeline is local first because session history is unusually sensitive. Raw transcripts, distillation, embeddings, graph data, and metadata can remain on one workstation instead of being copied into hosted AI systems with data practices outside the operator's control.

It is also intentionally distill-first: raw sessions become durable decisions, fixes, workflows, project context, and gotchas before graph or vector indexing.

flowchart TB
    subgraph historyRow[1. Recover the past privately]
        direction LR
        history[(Assistant history)] --> connector[Read-only connector]
        connector --> distill[Local LLM distillation]
        distill --> notes[(Resumable notes)]
    end

    subgraph memoryRow[2. Build reusable memory]
        direction LR
        cognee[Cognee extraction] --> memory[(Graph + vectors)]
        memory --> recall[Agent recall]
    end

    notes --> cognee

Loading

Choose a guide

I need to...	Start here
Understand the two phases, stores, and failure boundaries	Architecture
Configure models, embeddings, endpoints, or concurrency	Configuration
Smoke-test, launch, monitor, pause, resume, or back up	Operations
Understand how OpenCode sessions are reconstructed	OpenCode connector
Add Claude Code, Cursor, Cline, or another source	Adding a connector
Stop reasoning models from breaking JSON extraction	Thinking preamble
Review source isolation, scrubbing, and retention	Security and privacy

System properties

Property	What it means
Distill first	Raw transcript indexing is never the default
Read-only source	Connectors observe history and do not mutate it
Resume safe	Accepted notes reach disk before Cognee submission
Failure explicit	Errors retry; only model-confirmed JUNK is skipped
Evidence preserving	Exact paths, symbols, commands, and errors survive
Local first	LLM, embeddings, graph, vectors, and metadata can stay local

Connector status

Source	Status	Format
OpenCode	✅ Available	SQLite session, message, and part tables
Claude Code	🛠 Planned	Project JSONL
Cursor / Windsurf	🛠 Planned	Agent transcript exports
Cline / Continue	🛠 Planned	Extension storage / exports
Generic JSONL	🛠 Planned	Normalized external ETL