Architecture

Architecture

The system is the historical ingestion layer missing from forward-only agent memory. It has three resumable phases. Phase 1 converts noisy sessions into compact Markdown knowledge. Phase 2 adds those notes in idempotent batches. Phase 3 runs one bounded Cognee pipeline to build graph, vector, and relational representations. Every component can run locally, so raw histories do not need to cross a cloud API boundary.

End-to-end data flow

flowchart TB
    subgraph sourceLayer[Source layer]
        source[(Assistant history)]
        adapter[Source connector]
        source -->|read-only| adapter
    end

    subgraph phaseOne[Phase 1 - Distill]
        normalize[Normalize roles<br/>and bound fields]
        scrub[Scrub common<br/>secret formats]
        sample[Head + tail<br/>session sampling]
        transcripts[(var/transcripts<br/>model inputs)]
        llm[Local LLM]
        gate{Durable<br/>knowledge?}
        docs[(var/distilled<br/>Markdown)]
        skip[Junk skipped]

        normalize --> scrub --> sample --> transcripts --> llm --> gate
        gate -->|Yes| docs
        gate -->|No| skip
    end

    subgraph phaseTwo[Phase 2 and 3 - Add and Cognify]
        add[Batched Cognee add]
        cognify[One bounded<br/>Cognify run]
        classify[Classify + chunk]
        extract[Extract entities<br/>and relationships]
        summarize[Summarize chunks]
        embed[Generate embeddings]

        add --> cognify --> classify --> extract --> summarize --> embed
    end

    subgraph storage[Durable memory]
        metadata[(SQLite<br/>metadata)]
        graphStore[(Kuzu<br/>graph)]
        vectors[(LanceDB<br/>vectors)]
    end

    recall[Graph traversal<br/>and hybrid recall]

    adapter --> normalize
    docs --> add
    embed --> metadata
    embed --> graphStore
    embed --> vectors
    metadata --> recall
    graphStore --> recall
    vectors --> recall

Loading

Phase 1: distill

The source connector reconstructs a session in chronological order. The OpenCode connector opens SQLite with mode=ro and joins each content part to its parent message for role information.

Before the LLM

1. Discard non-content lifecycle events such as step-start and step-finish.
2. Bound text, reasoning hints, tool inputs, and tool output independently.
3. Replace common credential formats with redaction markers.
4. Sample long sessions from both ends: setup from the beginning, outcome from the end.
5. Skip sessions below a minimum substance threshold without an LLM call.

Distillation result

The model returns a Markdown work-history note. It preserves tasks attempted, projects and systems touched, investigations, artifacts, implementation details, findings, outcomes, failures, follow-ups, decisions, and reusable lessons. A decision is not required for retention. Exact JUNK is reserved for empty handshakes, pings, and sessions with no meaningful activity. Accepted notes are persisted to var/distilled/<source-id>.md before the source ID becomes distilled in progress.json. The normalized, bounded, scrubbed input shown to the model is persisted separately in var/transcripts/<source-id>.md; the OpenCode database remains the canonical source.

Phase 2: add

Persisted notes are sent to Cognee's ingestion-only add endpoint in bounded batches. Re-running this phase is safe: Cognee deduplicates identical content within a dataset by content hash. The connector therefore does not mirror Cognee's backend state in progress.json.

Phase 3: cognify

After all notes are present, the connector starts one Cognify run with explicit document and chunk concurrency. Cognee performs:

1. document classification;
2. chunking;
3. schema-constrained entity and relationship extraction;
4. chunk summarization;
5. embedding generation;
6. graph, vector, metadata, and provenance writes.

Storage map

Concern	Backend	Lifetime
Connector checkpoint	progress.json	Until verification / archive
Normalized model inputs	var/transcripts/	Audit / redistillation
Distilled source notes	var/distilled/	Until verification / archive
Dataset registry and metadata	SQLite	Cognee volume
Knowledge graph	Kuzu	Cognee volume
Vector index	LanceDB	Cognee volume
Container persistence	cognee_storage named volume	Until docker compose down -v

Resume state machine

stateDiagram-v2
    [*] --> absent
    absent --> distilled: useful note written
    absent --> skipped: exact JUNK / insufficient substance
    absent --> error: distillation failure
    error --> distilled: retry succeeds
    error --> skipped: retry confirms junk
    skipped --> [*]
    distilled --> [*]

Loading

Status	Meaning	Resume behavior
absent	Never attempted	Distill
error	Transient distillation failure	Distill again
skipped	Confirmed junk	Final
distilled	Model input and note exist on disk	No more distillation

Every run safely re-adds the selected persisted notes, then asks Cognee to Cognify the dataset. Cognee separately tracks content hashes and per-item graph and vector processing status, skipping completed work after a restart.

Skipped source IDs are retained in both progress.json and var/skipped.json. They can be reviewed or re-evaluated at any time with --reparse-skipped. Changing DISTILL_PROMPT_VERSION re-evaluates prior skips. Accepted notes stay stable unless --reparse-all is explicitly used against a new or empty target dataset; this prevents old and replacement content hashes from coexisting.

Failure boundaries

Failure	Recovery
Process killed during distillation	At most the active session repeats
Process killed after note write	Persisted note reloads on restart
Add batch fails	Service exits nonzero; launchd retries the idempotent add
Cognify fails or service restarts	Re-run Cognify; completed items are skipped
Cognee container recreated	Named volume preserves processed memory
Source grows during the run	Later incremental run or live plugin captures additions

Performance model

Phase 1 is dominated by LLM prefill and decode and can use multiple workers. Phase 3 is dominated by schema-constrained LLM calls; embedding is usually a small share of wall time. Beyond the GPU's memory-bandwidth limit, more concurrency lowers per-request speed without increasing total throughput.

The architecture deliberately avoids one background remember call per note. That pattern creates one pipeline run per document, defeating Cognee's per-run concurrency bound and overloading embedded SQLite, Kuzu, and LanceDB stores.

Measure completed sessions or documents per hour, not one request's displayed tokens per second.