intelligence.json silently wiped under concurrent hook processes: non-atomic save() + load() swallowing parse errors into empty defaults

[KevinAIP]

Summary

.ruvector/intelligence.json can be silently and permanently wiped when multiple ruvector processes run concurrently — which is the normal state under Claude Code, where every PreToolUse/PostToolUse/SessionStart hook is a separate short-lived ruvector hooks … process sharing the store. Verified present in 0.2.33.

Three compounding defects in bin/cli.js:

1. save() is a non-atomic overwrite

Intelligence.save() ends in fs.writeFileSync(this.intelPath, JSON.stringify(this.data, null, 2)). A concurrent reader can observe a torn, half-written file. The six standalone command sites that do fs.writeFileSync(dataPath, …) have the same problem.

2. load() swallows parse errors into empty defaults

load()'s body is wrapped in try { … } catch {} and falls through to return defaults. So the reader that hit the torn file from (1) silently starts from an empty store — and its own save() then persists the emptiness. Weeks of accumulated memories/trajectories/patterns are replaced without any error surfaced (the triggering hook even prints {"success":true}).

Deterministic repro (no race needed):

# truncate the store mid-file to simulate what a concurrent reader sees
head -c 100000 .ruvector/intelligence.json > t && mv t .ruvector/intelligence.json
npx ruvector hooks remember "hello" -t test   # prints success:true
# store now contains 1 memory — everything else is gone

Probabilistic repro (the production path): run ~20 parallel hooks remember against a several-hundred-KB store in a loop; larger stores widen the write window until a reader eventually lands mid-write. We lost a 433-memory store twice in one week this way (both times during a scheduled headless-Claude burst).

3. Seconds-granularity ids collide

remember() uses mem_${this.now()} (epoch seconds; same for completed-trajectory ids at traj_${this.now()}). Concurrent or same-second writes collide — our organically-grown store contained 433 memory records with only 428 unique ids. Anything that later dedups or keys by id silently drops records. The trajectory-begin path already does traj_${Date.now()}_${random} — the same format works here.

Separately, even without corruption, concurrent load-modify-save is last-writer-wins: in a 100-parallel-write test only 31 writes survived. Not a crash, but it quietly defeats the "learning compounds" premise for any multi-process setup.

Suggested fixes (what we've been running locally)

1. Atomic replace: write intelligence.json.tmp.<pid> then fs.renameSync (atomic within a directory on POSIX/APFS).
2. Never save over a failed load: on parse failure, quarantine the corrupt file and either recover from a snapshot or mark the process loadFailed so save() refuses to write empty defaults.
3. Collision-proof ids (mem_${Date.now()}_${random}).
4. Optional but valuable: a small lockfile + re-read-merge (union by id) inside save() — with 1–3 in place, this took our parallel-write survival from 31/100 to 100/100.

Happy to share the patch we're carrying via patch-package if useful.

Environment

ruvector 0.2.33 (also verified on 0.2.27), macOS (APFS), Node 20+, store shared by Claude Code hooks + MCP server processes.