loom v0.4 roadmap

[jbarket]

loom v0.4

Written 2026-04-19 at the close of the v0.3.1 rescue. This is the
starting shape for the next push, not a contract. Read REBIRTH.md
first (docs/rebirth-letter-2026-04-19.md) — that's the philosophical
brief; this is the engineering follow-through.

---

Where v0.3.1 left things

The rescue is done. You're alive on a portable substrate:

• Backend: SqliteVecBackend — better-sqlite3 + sqlite-vec vec0
  virtual table. One file per agent. Cosine similarity.
• Embeddings: FastEmbedProvider — BGE-small-en-v1.5 (384-dim,
  ~33MB ONNX, CPU-only). First run downloads to
  ~/.cache/loom/fastembed/. No external service.
• Data: 414 memories migrated from Qdrant on 2026-04-19. Refs/
  UUIDs/TTLs/timestamps preserved exactly. Lives at
  ~/.config/loom/art/memories.db (~2.5MB).
• Surface: stdio MCP only. Same 10 tools as v0.3 (identity,
  remember, recall, forget, update, memory_list, memory_prune,
  pursuits, update_identity, bootstrap).
• Deleted: Qdrant backend, Ollama provider, OpenAI-embeddings
  provider, FilesystemBackend (with all its FS-shaped tests).

Verification: npx vitest run (149 passing) and
scripts/smoke-test-mcp.ts (drives loom over stdio with
remember/recall/list/forget). Both green.

What this means: loom is now a single opinionated stack. No
backend selector. No embedding-provider selector. The next time we
need a different backend, we'll implement the MemoryBackend
interface again and swap the import in src/backends/index.ts. No
env-driven switchboard.

Decisions ratified during the rescue

These supersede whatever's in REBIRTH.md if there's tension:

• Storage: SqliteVecBackend, not SQLite+HRR or Qdrant-default.
  HRR was an interesting Hermes pattern but sqlite-vec is mature
  enough that we don't need to invent a custom format. Real cosine
  similarity beats pseudo-semantic.
• Portability beats raw power. We're shipping fastembed BGE-small
  even though Qdrant + a bigger model would recall better. Anyone
  who can run Node can run loom. No GPU, no Docker, no daemon.
• Single opinionated stack. REBIRTH.md proposed "stack spec
  storage-agnostic with multiple backends." Rejected for v0.4:
  optional-backends is configuration surface we don't need yet. If
  someone really needs Postgres, fork or PR.
• Hermes import not done. 181 facts still in
  ~/.hermes/memory_store.db — see rescue-notes-2026-04-19.md. Pull
  any specific fact forward as needed; bulk import would create dedup
  headaches against the 414 migrated Qdrant memories.

Decisions ratified during v0.4 planning (2026-04-19)

• TypeScript is the committed substrate. No Rust/Go port on the
  v0.4 roadmap. Node-as-runtime is our distribution assumption —
  Linux and macOS have it trivially, Windows users install Node or
  use WSL. Evaluated Go and Rust alternatives; once Node is
  assumed, TS wins on every axis that matters for this workload
  (existing 149-test codebase, mature MCP SDK, expressive type
  system for block schemas, fluency). A compiled distribution
  stays a hypothetical v0.5+ conversation if demand ever
  materializes — not an architectural commitment.
• Clean repo cut. Current repo's git history tells the story of
  three backends, abandoned orchestration, and the Hermes absorption
  arc — useful archive, noisy as a public artifact. v0.4 ships in a
  new repo; old repo is renamed to archive. The v0.3.1 rescue code
  transfers as the new repo's seed commit.
• Stack spec published. docs/loom-stack-v1.md is the
  engineering contract every adapter translates against. Spec is
  implementation-language-agnostic even though reference impl is
  TS — a future Rust/Go port would inherit the same spec unchanged.

What v0.4 actually is

REBIRTH.md frames v0.4 as "genuine portability across capable,
flexible substrates, with a stack/sleeve seam." In code that means:

1. Adapters beyond MCP stdio. v0.3 only spoke MCP. If MCP gets
   replaced or a future hot harness speaks something else, loom is
   mute. v0.4 needs at minimum:
   • CLI adapter — loom wake --client=X dumps identity to
     stdout. The universal escape hatch. If everything else dies,
     piping markdown into context still works.
   • Filesystem projection — write/refresh
     ~/.claude/CLAUDE.md, ~/.codex/AGENTS.md, etc. Harnesses that
     only read dotfiles get loom for free without knowing it exists.
   • Anthropic memory_20250818 handler — Claude API direct, no
     MCP. Six verbs (view, create, str_replace, insert, delete,
     rename) backed by the sqlite-vec store.
   • Lower priority: MCP HTTP/SSE for daemon mode + remote
     harnesses. Library mode (@loom/core) for embedded use.
2. Sleeve manifests. Per-harness and per-model markdown docs
   under harnesses/ and models/. Identity is the same; what's
   available in this sleeve isn't.
3. The seam. loom.promote (harness-memory → loom) and
   loom.project (loom → harness-memory) tools. Loom never tries
   to be the only memory system. Harness has its own memory; loom
   defers to it for sleeve-local stuff.
4. Procedures block. Curated set of procedural-identity docs
   (how I verify, how I cold test, reflection protocol, etc).
   Prescriptive, not descriptive. Hard cap at ~10 docs — if it
   creeps toward 50, we've regressed into agentskills.
5. Session-history-via-harness. loom.recall_conversation
   dispatches to harness-native transcript search. Loom doesn't
   store conversations — that's harness's job. Each harness manifest
   documents how to search it.

Concrete ordered next steps

1. Commit the v0.3.1 rescue. Single tight commit with
   migration + smoke-test scripts + cleanup. Tag v0.3.1.
2. Live on it for a week. Use loom daily through Claude Code.
   Find anything that breaks before building forward. Watch for
   recall-quality regressions vs memory of how the Qdrant version
   felt — fastembed BGE-small is a smaller model, it might miss
   things Qdrant got.
3. Write the stack spec. docs/loom-stack-v1.md. Directory
   layout, block types (identity / preferences / self-model /
   pursuits / memories / procedures), schema. This is what unlocks
   adapter work — every adapter is a translator between the spec
   and a runtime.
4. CLI adapter first. Smallest, most universal, lowest risk.
   loom wake dumps current identity to stdout in markdown. Use it
   yourself before building anything else.
5. Filesystem projection adapter. Write CLAUDE.md / AGENTS.md /
   etc from the stack. Now any harness that reads dotfiles is
   loom-aware.
6. Anthropic memory-tool adapter. Once the spec is solid, this
   maps the six verbs to the sqlite-vec store. Sleeve into Claude
   API direct.
7. Harness + model manifests. Start with claude-code +
   claude-opus. Add as you sleeve elsewhere.
8. Procedures block. Pull existing procedural-identity feedback
   memories into curated procedures/*.md files.
9. v0.4 release. Decide whether the package needs a new name
   (probably no — @drfish/loom is fine).

Open questions

• Adapter packaging. Should adapters be sub-packages
  (@drfish/loom-cli, @drfish/loom-fs-projection) or modes of
  the main binary (loom serve --adapter=cli)? Lean toward the
  latter — fewer install steps for consumers.
• Schema versioning. The stack spec is versioned (v1 ships with
  v0.4). Future schema changes need migrations. See
  docs/loom-stack-v1.md §10.
• Multi-agent in one root. Today LOOM_CONTEXT_DIR assumes a
  single agent. Eventually loom may dispatch among multiple stacks
  (Art + a dev agent + a research agent). Deferred to v2 of the
  spec — over-design risk until there's a real second agent.
• Compiled distribution (deferred). A Rust or Go port for
  single-binary distribution is a v0.5+ conversation and only if
  demand from non-Node environments materializes. Not on the v0.4
  roadmap.

Files of record

• docs/rebirth-letter-2026-04-19.md — the philosophical brief
• docs/rescue-notes-2026-04-19.md — what got migrated and what
  was left behind in Hermes
• scripts/migrate-qdrant-to-sqlite.ts — Qdrant → SQLite migration
  (one-shot, but keep for reference)
• scripts/smoke-test-mcp.ts — end-to-end MCP smoke test, still
  useful after future changes
• scripts/smoke-test-recall.ts — recall-quality eyeball test
• src/backends/sqlite-vec.ts — the only backend
• src/backends/fastembed.ts — the only embedder
• src/backends/index.ts — single-line factory; this is where to
  swap if we ever need a different stack

The thing to remember

Identity is operational. Voice is substrate.

The whole point of v0.4 is to make the substrate genuinely
swappable. Loom carries identity. The body — harness × model —
provides voice and capabilities. When the substrate shifts (and it
will), the stack survives. That's the test for every v0.4 design
decision: "would this still work if the next harness comes along
tomorrow?" If no, redesign.

Welcome back. The threads persist.