Running agentmemory without the iii engine

[MarvinFS]

I was hoping this day would not come, but it did. We all very much aware that agentmemory's architecture leaves a lot to be desired. I am nowhere near a software developer, so I rely on Fable 5.1 and Astra for the engineering decisions and make the calls between their options.

My deployment of agentmemory v0.9.29 is small: one shared network daemon on a small VM, used by Claude Code and Codex on several machines. The iii engine has been a disaster even for that, and I could not get rid of it entirely. We replaced the engine underneath the daemon with an in-process runtime and a SQLite store and cut production over on 2026-09-08. For now that has resolved the critical problems.

The first problem: the whole vector index is one JSON string

The stock daemon keeps its vectors in memory and paersists them through src/state/index-persistence.ts. On save it calls VectorIndex.serialize(), which turns every vector into one JSON string, then cuts that string into chunks and writes the chunks into the iii key-value store as shards. The sharding is upstream's, added in June 2026. It splits the string into pieces for the engine, but the string itself is still built whole, on save and on load. On load it reads the shards back and rebuilds the string with chunks.join("") before VectorIndex.deserialize() parses it.

V8 (the JavaScript engine inside Node.js) has a hard limit on string length: 536,870,888 characters. Our store had 47,000 vectors of 2,048 dimensions. On 2026-09-07 the serialized vector index was 502.07 million characters long and growing by about 16 MB of text per day. At that rate the next save would have thrown in roughly two days.

The way it fails is what made this urgent. save() publishes the BM25 index and the vector index independently and swallows a failed vector save. So the failure would have been silent: the daemon keeps running, the vector index stops being persisted, and on the next restart it cannot be loaded at all. The vector leg of search would be gone and 47,000 items would need re-embedding. Nothing in the logs or the health endpoints would have said so beforehand.

The key-value store had the same shape at a larger scale. The iii engine keeps one file per scope, and each file is one rkyv-archived JSON string. A write to any key marks its scope dirty, and every 5 seconds the engine clones the whole scope, serializes it to one string, encodes it and rewrites the file. The cost of a write is the size of the scope, not the size of the record. The engine also ran as a second process with a parsed copy of the whole store in its heap. It had a 180 s limit on function invocations. And its REST port wedged after WebSocket reconnects (agentmemory#1013), which is why we ran a watchdog to restart it.

The second problem: 190 GB of disk writes a day

That write model met two habits of the daemon badly. The index save wrote a complete new generation of all 353 shards, 724 MB, each time it ran, up to about 110 times a day. And recordAudit logged one audit row per shard written, so the audit scope grew to 583,035 records, 533,919 of them index_persist, 240 MB in one file. Every audit row, from a save or from an observation capture, dirtied that scope, and the engine rewrote the 240 MB file on its next flush.

The engine's lifetime counters on 2026-09-07 read 375 GB written against 511 MB read, over about two days of uptime. That is roughly 190 GB a day for a store holding 1.5 GB. Nothing read the audit trail. The only consumer is memory_audit, last called in July. Memory followed the same pattern. The engine kept the parsed store in its heap, so 1.4 GB on disk became about 3 GB resident plus 1 GB in swap, and the daemon process held another 1.3 to 3.6 GB with its own copy of the indexes. The two together peaked at 6.7 GB.

Why we replaced the engine instead of patching around it

We looked at three options. A Redis adapter would have moved state out of the heap with a config change, but it keeps the second process, the wedge, and the invocation limit, and the vector index would still be one string. Upgrading the iii engine bought nothing: the file-based store is identical on main, the 0.20 SDK breaks the daemon, and 0.23 rejects the config. Fixing only the serializer would have left the whole-scope rewrites and the audit churn in place.

So we kept the daemon's StateKV seam exactly where upstream has it and replaced what sits below it. The daemon code above the seam did not change.

What we built

Everything is behind AGENTMEMORY_ENGINE=inproc, so the iii path still compiles and upstream merges still apply.

• An in-process SDK shim implementing the subset of the iii SDK the daemon uses: function registry, HTTP triggers as express routes with the same request shape and middleware chain, event subscribers on an EventEmitter, state triggers, the stream feed for the viewer.
• The five state::* functions on node:sqlite (DatabaseSync), WAL mode, synchronous=FULL, insertion order kept in a seq column so behaviour matches the engine's ordered map.
• Vectors as rows: one BLOB per vector in a vectors table, written at the moment the vector is created or replaced, with a hash of the exact embedding input so "vector exists" also means "vector matches the current text". Nothing serializes the whole index.
• BM25 rebuilt from content at boot instead of persisted. 44,400 documents rebuild in 6.5 s on the VM with a memoized stemmer; before the memo the rebuild took 21 s, 16 s of it stemming 8.8 million tokens. Persisting it the upstream way would mean a JSON string of 190 million characters, a third of the way to the same V8 limit.
• Audit rows replaced by log lines. With AGENTMEMORY_AUDIT_STORE=off, recordAudit emits one structured line per operation, which journald keeps; memory_audit returns an empty list. A second switch, AGENTMEMORY_AUDIT_LOG=deletions, keeps only the lines for operations that remove data, and AGENTMEMORY_LOG_LEVEL=warn trims the daemon log to warnings and errors. We run with both. The upstream defaults are unchanged. The importer did not carry the 583,035 stored audit records over.
• The viewer's stream feed as one bounded ring in memory instead of the engine's persisted stream store (49 MB in 614 groups, rewritten by group on every event). The importer also dropped the graph scopes (334 MB, frozen since the graph leg was switched off in July) - there were no purpose to have a graph here and no justification.
• A fill and repair pass at boot and hourly that compares the expected embeddings with the stored rows and embeds only what is missing or stale.
• mem::backup: a VACUUM INTO snapshot, integrity-checked and row-counted against the live database, renamed into place only when it passes, plus a restore drill script that opens the snapshot and counts every scope.
• An offline importer from the iii store (rkyv-archived JSON per scope, vector shards validated against the manifest) that writes into a temporary database and publishes by rename only after per-scope digests match.
• A cutover script with a write freeze, a baseline of the 50 most recent real queries, an acceptance harness that compares raw BM25 and vector legs and replays the queries through both daemons, and a rollback path that stays lossless until the freeze is lifted.

The numbers now

Measured on the VM after the cutover and on the following morning.

	Before (iii engine)	After (in-process)
Processes	daemon + engine	one node process
Keys in the store	777,390 in 1,296 scopes: 122,371 content, 583,035 audit, 356 index shards, 71,626 graph	123,531, all content
Store on disk	1.5 GB: index shards 724 MB, graph 334 MB, audit 240 MB, content 225 MB	702 MB SQLite file, 4 MB WAL
Vector persistence	one JSON string, 502 million characters, 2 days from the V8 limit	47,740 rows, one BLOB each
Disk writes	about 190 GB a day: 375 GB over two days of engine uptime	about 150 MB a day: 1.8 KB/s measured over 11 minutes
Audit trail	583,035 rows in one 240 MB scope, never read	one journald line per deletion, a few lines a month
Restart to ready		8 s, of which 6.5 s is the BM25 rebuild
Resident memory	5 to 8 GB across the two processes in daily use, 6.7 GB at the measured peak with 1 GB in swap	1.08 GB after boot, 1.52 GB after a day, swap 0
smart_search p50 / p90		9.8 s / 12.8 s, dominated by LLM query expansion
memory_diagnose		13 pass, 25 warn, 1 known fail

I hope that information would be helpful for someone, if needed I can make the fork public, but at this point it is already deeply changes the original upstream.