your data exports become a semantic knowledge graph. drop spotify history, claude conversations, photos, text files. a hierarchical swarm of gemini agents extracts every entity, relationship, claim, and event into a 15 table evidence backed memory layer.
the backend is the product.
llms have no memory. every conversation starts from zero. every agent you spin up knows nothing about you. memopal fixes that.
it builds a persistent semantic context layer from your actual data. not summaries, not embeddings of chat logs. structured knowledge with full provenance. entities, relationships, claims, evidence, confidence scores, all queryable via rest api.
the point is not the ui. the point is that any autonomous agent, claude code, cursor, a cron job, a slack bot, a custom tool calling loop, can hit /api/entities or /api/search and instantly have structured context about who you are, what you work on, who you know, what you care about. agents that can remember you without you repeating yourself.
memopal is the memory layer for agents that don't have one.
the core idea: the orchestrator cannot write to the database. it has zero write tools. it must delegate to sub agents, which themselves spawn further sub agents. this forces genuine n depth agent trees instead of a single llm with tools pretending to be agentic.
graph TD
U[user drops files] --> O
O["orchestrator\ngemini 3.1 pro, 30 steps\nno write tools, must delegate"] -->|delegates| PA["processArchive\nworker, 20 steps"]
O -->|delegates| PD["processDirectory\nworker, 20 steps"]
O -->|delegates| EE["extractEntities\nworker, 10 steps"]
O -->|delegates| RP["reasonAboutProfile\norch tier, 15 steps"]
PA --> PF["processFile x n\nleaf, 10 steps"]
RP --> IL["investigateLead\nworker, 10 steps"]
PA -->|writes| DB[(sqlite\n15 tables)]
PD -->|writes| DB
PF -->|writes| DB
EE -->|writes| DB
RP -->|writes| DB
IL -->|writes| DB
style O fill:#1c1a00,stroke:#fbbf24,color:#fbbf24
style PA fill:#001a0a,stroke:#4ade80,color:#4ade80
style PD fill:#001a0a,stroke:#4ade80,color:#4ade80
style EE fill:#001a0a,stroke:#4ade80,color:#4ade80
style RP fill:#1c1a00,stroke:#fbbf24,color:#fbbf24
style PF fill:#001a1a,stroke:#22d3ee,color:#22d3ee
style IL fill:#001a1a,stroke:#22d3ee,color:#22d3ee
style DB fill:#1a0a0a,stroke:#f87171,color:#f87171
style U fill:#111,stroke:#888,color:#ccc
three model tiers run in parallel. orchestrator on gemini 3.1 pro for planning and delegation. workers on gemini 3 flash for file processing and entity extraction. leaf agents on gemini 3.1 flash lite for individual file parsing. each tier has its own step budget and tool set. agents call 5 to 10 tools per step via promise.all.
this project would not work without either of these.
ai sdk v6 is what makes the agent architecture possible. generateText with tool definitions gives each sub agent its own autonomous loop with a step budget. tools execute as real async functions, not json stubs. when the orchestrator calls processArchive, that tool internally spins up its own generateText loop with its own tools, creating genuine nested agent execution. the onStepFinish callback streams every tool call to the trace system in real time. no other framework gives you composable agent loops that nest like this without fighting the abstraction.
gemini is what makes it affordable and fast enough to actually run. memopal processes thousands of artifacts per session. a single spotify export can have 10,000+ listening events. a claude conversation export can have hundreds of threads. you need a model that can handle massive context windows cheaply and run many parallel calls without rate limit hell. gemini 3.1 pro handles orchestration with 1m context. gemini 3 flash processes files at sub second latency. gemini 3.1 flash lite parses individual artifacts for pennies. gemini embedding 2 generates vectors for semantic search across the entire graph. running this on gpt 4 or claude would cost 10x more and hit rate limits before finishing a single export. the three tier model strategy only works because gemini offers three performance tiers that map directly to orchestrator, worker, and leaf roles.
every extracted fact is a claim. every claim links to evidence units. every evidence unit links to the source artifact. every artifact links to the original file. full provenance chain, nothing is asserted without receipts.
graph LR
SF[sourceFile] --> A[artifact] --> EU[evidenceUnit] --> CE[claimEvidence] --> C[claim]
C --> E[entity]
E --> RE[relationshipEdge]
A --> EV[embeddingVector]
style SF fill:#1a0a0a,stroke:#f87171,color:#f87171
style A fill:#1a0a0a,stroke:#f87171,color:#f87171
style EU fill:#1a0a0a,stroke:#f87171,color:#f87171
style CE fill:#1a0a0a,stroke:#f87171,color:#f87171
style C fill:#1a0a0a,stroke:#f87171,color:#f87171
style E fill:#1a0a0a,stroke:#f87171,color:#f87171
style RE fill:#1a0a0a,stroke:#f87171,color:#f87171
style EV fill:#1a0a0a,stroke:#f87171,color:#f87171
15 tables in sqlite with wal mode via drizzle orm. import sessions track upload batches. source files track raw inputs. artifacts are parsed data units (a chat message, a song play, a photo). entities are people, places, orgs, projects with confidence scores and status (candidate until user confirms). claims are typed facts about entities. relationship edges connect entities. embedding vectors enable semantic search via gemini embedding 2. trace events log every agent step for the real time canvas. proposal batches gate everything behind human approval.
gemini embedding 2 vectors stored per entity. cosine similarity search over the full graph.
curl "http://localhost:3000/api/search?q=machine+learning+projects&limit=5"the memory layer is fully queryable. plug it into any agent. a coding assistant that knows your stack. a scheduling bot that knows your team. a research agent that knows your interests. the api is the product.
| endpoint | method | description |
|---|---|---|
/api/upload |
POST | multipart file upload, creates import session |
/api/ingest |
POST | triggers orchestrator on import session |
/api/search |
GET | semantic search over entities (?q=...&limit=N) |
/api/entities |
GET | all extracted entities with metadata |
/api/events |
GET | all timeline events |
/api/claims |
GET | all claims with subject entity names |
/api/relationships |
GET | all relationship edges |
/api/proposals |
GET/POST | approval queue, accept or reject |
/api/traces |
GET | sse stream of agent execution |
sequenceDiagram
participant U as user
participant API as next.js api
participant O as orchestrator
participant W as workers
participant DB as sqlite
U->>API: POST /api/upload (files)
API->>DB: create importSession + sourceFiles
U->>API: POST /api/ingest
API->>O: orchestrator.generate()
loop 30 steps max
O->>W: processArchive / processDirectory
W->>DB: writeArtifact, writeEntity, writeClaim
O->>W: extractEntities
W->>DB: writeEntity, writeRelationship
O->>W: reasonAboutProfile
W->>DB: writeClaim, writeEvidence
end
O->>DB: createProposalBatch
API-->>U: SSE traces (real time)
U->>API: POST /api/proposals (approve/reject)
the visualizations exist to make the backend legible, not the other way around.
galaxy 3d renders entities as star clusters in a procedural spiral. confidence determines proximity to center. relationships are particle trails. the agent canvas streams every tool call and delegation chain in real time via sse into a react flow graph. profile view shows evidence backed claims with confidence bars and reasoning traces. timeline reconstructs life events grouped by era. approval queue gates everything behind human review.
| layer | tech |
|---|---|
| ai | gemini 3.1 pro / 3 flash / 3.1 flash lite, ai sdk v6 |
| embeddings | gemini embedding 2, cosine similarity search |
| framework | next.js 16, react 19 |
| 3d | three.js, react three fiber |
| graphs | react flow |
| database | sqlite + drizzle orm, 15 tables, wal mode |
| streaming | server sent events |
| ui | shadcn, tailwind, framer motion, geist |
no write tools on orchestrator. forces real delegation. the orchestrator plans and coordinates. sub agents do the actual extraction and writing. this creates genuine agent trees, not a wrapper around tool calling.
confidence propagation. every entity, claim, event, relationship carries a 0 to 1 confidence score from the moment of extraction through to the approval ui. nothing is asserted as fact without a score.
evidence chains. claims trace back to evidence units, evidence units trace back to artifacts, artifacts trace back to source files. you can always answer "why does the system think this."
human in the loop. nothing enters the knowledge graph without user approval. agents propose, humans confirm. the approval queue is not a formality, it is the trust boundary.
parallel tool execution. agents call multiple tools per step via promise.all. a single orchestrator step can spawn 4 sub agents simultaneously. a single worker step can read 10 files and write 10 artifacts in parallel.
git clone https://github.com/qtzx06/memopal.git
cd memopal
bun install
cp .env.example .env.local
# add GOOGLE_GENERATIVE_AI_API_KEY
bun run devopen localhost:3000, press space, drop files.
supports any file. zip archives, json, csv, txt, md, images, pdfs. the agents figure out what it is and extract what matters. spotify exports, claude conversations, google takeout, imessage backups, notes, photos, whatever you have.
built at zero to agent: vercel x deepmind hackathon nyc, march 21 2026