Skip to content

feat: wiki single-source-of-memory replaces FTS5 memory tools#83

Merged
UmarFarooq75 merged 6 commits into
mainfrom
dev
May 10, 2026
Merged

feat: wiki single-source-of-memory replaces FTS5 memory tools#83
UmarFarooq75 merged 6 commits into
mainfrom
dev

Conversation

@UmarFarooq75
Copy link
Copy Markdown
Collaborator

@UmarFarooq75 UmarFarooq75 commented May 9, 2026

Summary

Switches daemora's memory model to a Karpathy-style wiki — persistent LLM-maintained markdown under data/wiki/ — replacing the separate FTS5 notebook the agent queried via memory_save / memory_recall. Single source of truth, accessed through the agent's existing filesystem tools.

Why

The agent had two parallel paths for "remember a fact" (memory_save) and "look up a fact" (memory_recall) on top of an FTS5 table that re-derived synthesis on every query. That setup encouraged the agent to hallucinate which path to use and rediscover context every recall. The wiki keeps the synthesis already built and lets the same filesystem tools the agent already has carry the load.

Architecture

  • data/wiki/{projects,people,topics,decisions}/*.md — synthesis pages
  • data/wiki/index.md — table of contents
  • data/wiki/log.md — append-only event ledger (every memory save + gallery change writes one line)
  • data/wiki/.sync-cursor — heartbeat sync watermark

Phase 1 — skeleton + write hooks

  • New src/wiki/WikiLog.ts — idempotent skeleton + atomic append (POSIX O_APPEND).
  • MemoryStore.save() and FileProjectStore mutations (create / updateMeta / delete / addFile / removeFile) each append one structured line to log.md so synthesis turns get reliable signal.

Phase 2 — idle maintenance turn

  • Heartbeat gets a third mode (wikiSyncCheck) on a 10 min cadence (WIKI_SYNC_INTERVAL_MINUTES env override).
  • Reads only the unsynced tail of log.md, drops trailing partial lines, hands the agent a 16KB-capped delta + the new cursor, and the same main agent folds events into pages with read_file / write_file / edit_file.
  • Skips when the user has been active within the last 5 min (SessionStore.lastUserActivityAt() tracks user-role message timestamps).
  • Best-effort: cursor only advances after the agent writes it back, so a failed turn just retries on the next tick.

Phase 3 — drop the duplicate path

  • Remove memory_save and memory_recall from the tool registry.
  • MemoryStore stays wired so existing rows remain queryable from internal code; the agent simply no longer sees a memory tool surface.

SOUL.md

  • Replaced the rule-list ## Memory section with a principle-led ## Wiki — your single source of memory section: page health (50–200 lines, cap 350), conflict handling (in-place blockquotes), source provenance, idle-maintenance contract.
  • Fixed stale searchMemory / writeMemory references in the teamTask section.

Test plan

  • npm run typecheck clean (verified locally)
  • First boot creates data/wiki/ skeleton with index.md, log.md, four subdirs
  • Calling any memory-affecting path (memory save via internal code, gallery file add, project create) appends one event line to log.md
  • Heartbeat tick logs wiki sync — enqueuing maintenance turn when log.md has unsynced content and the user has been quiet for >5 min
  • Memory tools (memory_save / memory_recall) no longer surface to the agent
  • No runtime crash with empty data/wiki/ (skeleton creation is idempotent)

CI npm ci was failing — package-lock.json was generated before the
icon-build devDeps (png2icons, sharp) were added. Regenerated lockfile
to include the missing entries.
…ruth

Switching daemora's memory model to a Karpathy-style wiki: persistent,
LLM-maintained markdown under data/wiki/ instead of a separate FTS5
notebook the agent queries via memory_save / memory_recall. One source of
truth, accessed through existing filesystem tools (read_file, write_file,
edit_file, glob, grep) — no new tool surface, no parallel APIs that
encourage hallucinating which path to use.

Architecture:
- data/wiki/{projects,people,topics,decisions}/*.md  — synthesis pages
- data/wiki/index.md                                  — table of contents
- data/wiki/log.md                                    — append-only event ledger
- data/wiki/.sync-cursor                              — heartbeat sync watermark

Phase 1 — skeleton + write hooks
- New src/wiki/WikiLog.ts: idempotent skeleton + atomic append.
- MemoryStore.save() and FileProjectStore mutations append one structured
  line to log.md so every fact and gallery change shows up for synthesis.

Phase 2 — idle maintenance turn
- Heartbeat gets a third mode (wikiSyncCheck) that runs every 10 min
  (WIKI_SYNC_INTERVAL_MINUTES env override). Reads the unsynced tail of
  log.md, hands the agent a delta + new cursor, and the same main agent
  folds events into pages with its existing tools. Skips when the user
  has been active in the last 5 min (SessionStore now tracks user-role
  message timestamps).

Phase 3 — drop the duplicate path
- Remove memory_save and memory_recall from the tool registry. The agent
  now has exactly one place to read and write memory. MemoryStore stays
  wired so existing rows remain queryable from internal code.

SOUL.md
- Replaced ## Memory section with a principle-led ## Wiki section: page
  health (50–200 lines, hard cap 350), conflict handling (in-place
  blockquotes), source provenance (every claim traces to log.md or
  data/file-projects/), idle-maintenance contract.
- Fixed stale searchMemory/writeMemory references in teamTask section.
…cs, conventions

The Wiki section now opens with what the wiki *is* (a small interlinked
book of synthesis that compounds over time) instead of jumping to where
files live. Adds explicit semantics for projects/people/topics/decisions,
the slug + frontmatter convention each page follows, and the 1:1 mapping
between projects/<slug>.md and data/file-projects/<slug>/. Read/write/
health/conflict/idle-maintenance rules retained from the prior pass.
…llback rules

- Add ensurePlaywrightChromium() helper that runs `npx -y @playwright/mcp@latest install-browser chrome-for-testing` so the browser binary is on disk before the MCP server is asked to drive it. Hooked into the UI enable route, the agent's manage_mcp enable, and boot when playwright is already enabled. No more "Browser chrome-for-testing is not installed" mid-task.
- Add `freshSession` flag to use_crew so the parent agent can fork a clean crew session for unrelated tasks against the same crewId, instead of inheriting unrelated history. Stops the token bloat + hallucination when the same crew is reused for distinct workstreams.
- SOUL.md + crew system prompt: keep generated/temp files under data/. Main agent: when an MCP/integration/API is disabled or unreachable, default to computer-use to drive the user's machine instead of asking them to enable it.
- Bump to 1.0.0-alpha.7.
Resolved package-lock.json version-string conflict by keeping the dev-side bump (1.0.0-alpha.7). All other lockfile content (icon-build devDeps from #82, computer-use MCP deps from #81) auto-merged cleanly.
@UmarFarooq75 UmarFarooq75 merged commit 5ae23df into main May 10, 2026
4 checks passed
UmarFarooq75 added a commit that referenced this pull request May 11, 2026
origin/main got a squash-merge of #83 which dropped the SHAs of dev's wiki commits, so dev replays them as 'new' content. Resolved by keeping ours: alpha.8 version, my stop_crew rule, and the existing 'focus on it as well' phrasing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant