Skip to content

RuLake-inspired feature roadmap (Phases 0-3)#1

Merged
shaal merged 10 commits intomainfrom
rulake-roadmap
Apr 24, 2026
Merged

RuLake-inspired feature roadmap (Phases 0-3)#1
shaal merged 10 commits intomainfrom
rulake-roadmap

Conversation

@shaal
Copy link
Copy Markdown
Owner

@shaal shaal commented Apr 24, 2026

Summary

Implements the full RuLake-inspired feature roadmap documented in
docs/plan/rulake-inspired-features.md. 10 commits land seven
user-visible features plus the foundations + tour-integration +
shareable-archive polish that ties them together.

Every feature is flag-gated (default-off) or default-on-but-
behaviour-neutral. A casual visitor to the running app sees the new
ELI15 chapters and the observability panel; demo-hunters can unlock
the rest via URL flags.

Commits (in land order)

# Commit Phase What it adds
1 c33003e 0 Foundations — ArchiveSnapshot schema, xxHash32 util, 4 bridge stubs, 7 chapter placeholders
2 b8b2f60 1D Content-addressed dedup + hash-keyed lineage; DAG viewer ×N badges
3 7b95af0 1B RaBitQ 1-bit quantizer + Hadamard rotation; 30× memory shrink; live recall@10 = 0.9060
4 ee36b44 1A Warm-restart bundles (archive export/import via CompressionStream); ?snapshots=1
5 c9df280 1C Fresh / Eventual / Frozen consistency modes + radio row + tick strip
6 bf02701 2A Federated search (Euclidean + Hyperbolic) with k' = k + ⌈√(k·ln S)⌉ over-request + GNN rerank
7 ae32aba 2B Cross-tab live training via BroadcastChannel; lockless-by-content-hash
8 0c51aaa 3A Observability dashboard + per-stage timings; fills the last `not implemented` stub
9 bebba81 3B ELI15 tour integration pass — 4 pedagogical bands + cross-links
10 93c9dc4 3C Shareable archive URLs + placeholder-only community gallery

What users see

By default (no flags):

  • 7 new ELI15 chapters (~5,400 words of teaching content)
  • Registry re-ordered into 4 pedagogical bands; 7 chapters carry `related` cross-links
  • `⏱ Where the time goes` observability panel (collapsible, telemetry only)
  • `🧊 Consistency modes` radio row (Fresh default = unchanged behaviour) + tick strip
  • `🌐 Federation` checkbox (unchecked default)

Behind URL flags:

  • `?snapshots=1` → Export/Import archive + Copy-shareable-link + Import-from-URL + 1-entry gallery (placeholder)
  • `?consistency=fresh|eventual|frozen` → preset a consistency mode at boot
  • `?federation=1` → federation on + split-screen Euclidean/Hyperbolic viewer
  • `?crosstab=1` → 🔗 N-peers pill + live cross-tab archive sync
  • `?snapshots=1&archive=` → auto-import a remote archive at boot

Stats

  • ~9,400 lines of feature code across 10 commits
  • 8 live browser test harnesses under `/tests/`, each with PASS/FAIL in-DOM output
  • 8 ELI15 chapters rewritten from Phase 0 stubs to ~500-900 words each
  • 4 subagent swarm waves (1B+1D parallel, 1A seq, 1C seq, 2A seq, 2B seq, 3A+3B parallel, 3C seq)
  • Zero merge conflicts, zero rollbacks, zero remaining `not implemented` stubs in ruvectorBridge.js

Known follow-ups

  • UI discoverability pass (surface feature toggles in UI instead of URL-only); proposed as next PR.
  • F1 quantization is library-only — not wired to the archive hot path yet. `?quant=1` does not exist.
  • F5 content-addressed dedup runs on the cross-tab receive path and on import; not yet on the single-tab same-session `archiveBrain` hot path.
  • Community archive gallery contains exactly one `about:blank` placeholder — real URLs require explicit review before publishing.
  • No default-on flag-flips. All new behaviour remains opt-in.

Test plan

  • Open `https://vectorvroom.pages.dev/AI-Car-Racer/index.html\` (or local `python3 -m http.server 8765`) — confirm clean boot, no new console errors, ELI15 drawer shows the 7 new chapters in the re-ordered list.
  • Navigate to each of the 8 `/tests/*-smoke.html` harnesses — confirm PASS for all claims.
  • `?snapshots=1` → Export archive, reload the tab, Import archive; confirm top-1 id is byte-identical to pre-export.
  • Open two tabs with `?crosstab=1` → confirm 🔗 1 peer on each; archive a brain in one, confirm it arrives in the other.
  • `?federation=1` → confirm split-screen viewer renders, formula row shows live k' number.
  • `?consistency=frozen` → confirm Frozen radio selected at boot; archive a new brain, confirm it doesn't appear in `recommendSeeds` output.

See `docs/plan/rulake-inspired-features.md` for the full plan, phase-by-phase status, and decisions log.

Summary by CodeRabbit

  • New Features

    • Added snapshot save/restore capability for warm-restart sessions
    • Added federated search across multiple index types with union/reranking
    • Added consistency modes (fresh/eventual/frozen) with optional query caching
    • Added cross-tab live training synchronization via BroadcastChannel
    • Added 1-bit quantization for memory-efficient vector storage
    • Added observability panels displaying retrieval pipeline timings
    • Added content-addressed brain deduplication and URL-based sharing
  • Documentation

    • Added comprehensive chapters on archives, federation, consistency modes, quantization, and cross-tab features
  • Tests

    • Added smoke tests validating new features across pipelines

shaal added 10 commits April 24, 2026 12:25
… stubs

Foundations for the RuLake-inspired feature roadmap
(docs/plan/rulake-inspired-features.md). Phase 0 is deliberately
boring: define shared shapes + cut extension points so the four
Phase 1 swarm tasks never fight over ruvectorBridge.js.

What this ships:
- archive/snapshot.js — ArchiveSnapshot v1 shape + validateSnapshot()
- archive/hash.js — xxHash32 over Float32Array brains; 8-char hex ID
- ruvectorBridge.js — exportSnapshot / importSnapshot /
  setConsistencyMode / getConsistencyMode / getIndexStats. Stubs
  throw 'not implemented' with owning-phase labels; importSnapshot
  already validates at the boundary so callers can test wiring.
- eli15/chapters/*.js — 7 placeholder chapters (warm-restart,
  quantization, consistency-modes, content-addressing, federation,
  cross-tab-federation, where-the-time-goes), each with comingSoon:
  true and a real ELI15 hook.
- eli15/index.js — registry entries for the 7 new chapters.

What this deliberately does not do: no behaviour change, no new
feature. The bridge imports validateSnapshot at module scope but
doesn't call it at boot; chapters are lazy-loaded.

Validated via agent-browser: app boots cleanly with no new console
errors, all 7 chapter IDs returned by ELI15.listChapters(), two
chapters rendered end-to-end, stub contracts (5 exports, correct
error messages, consistency default = 'fresh'), hash determinism +
1-ppm sensitivity.
F5 from docs/plan/rulake-inspired-features.md. Two brains with
identical flattened weights now collapse to a single canonical node
in the lineage DAG — the DAG viewer shows a "×N" badge on
rediscovered brains, and a duplicateRatio stat is exposed for the
training panel.

What this ships:
- archive/dedup.js — in-memory Map<hash, {firstSeenId, dupCount}>
  backing maybeInsert() / stats() / has() / get() / _debugReset().
  Backed by archive/hash.js's xxHash32 from Phase 0.
- lineage/dag.js — new addBrainWithFlat(flat, meta, fallbackId)
  hashes the flat vector and dedups at insert; legacy addBrain()
  signature unchanged (no behaviour change on existing callers).
  dedupeStats() exposed for the viewer + training panel.
- lineage/viewer.js — renders "×N" badge on nodes with
  duplicateCount > 0; tooltip reads "×N seen".
- brainExport.js — export rows carry the hash; new programmatic
  importBrain(row) short-circuits on dedup hit and returns
  { skipped: true, reason: "duplicate" }. Legacy export/import
  path still works if late-bound hooks aren't wired.
- eli15/chapters/content-addressing.js — full chapter replacing
  the Phase 0 coming-soon stub.
- tests/dedup-smoke.html — standalone harness, PASS/FAIL in DOM.

Not wired into ruvectorBridge.archiveBrain on the GA hot path — that's
a later integration slice. Module is callable but opt-in.

Validated via agent-browser: smoke harness prints PASS on all 3
claims (maybeInsert idempotent, dedupeStats shape + counts, importBrain
re-import leaves node count unchanged). Main app still boots cleanly
with no new console errors after the lineage/dag.js + viewer.js +
brainExport.js edits.
F1 from docs/plan/rulake-inspired-features.md. Ships a standalone
quantizer that shrinks brain vectors ~30× (976 B float → 32 B packed,
or 96 B including a 16-float residual) with recall@10 = 0.9060 on the
brain-lineage-cluster workload. Not wired into the hot path — Phase 2A
federation does that.

What this ships:
- quantization/hadamard.js — in-place iterative Fast Walsh-Hadamard
  Transform, pads to next power of 2 (244 → 256).
- quantization/rabitq.js — quantize(flat) → {packed: Uint32Array,
  residual: Float32Array[16]}; hammingDistance via popcount on XOR;
  estimatedCosine uses the Hamming/π identity.
- quantization/index.js — public surface; Phase 2A consumes from here.
- quantization/viewer.js — mountViewer(container, samples[]) plain-DOM
  side-by-side float heatmap / bit-packed view / true-vs-estimated
  cosine scatter plot / memory meter. No framework.
- eli15/chapters/quantization.js — full chapter replacing the Phase 0
  coming-soon stub. Covers the SimHash analogy, why Hadamard beats
  random projections, the 30× memory win, and explicitly flags that
  hyperbolic HNSW is out of scope for F1 (sign-bit math is Euclidean).
- tests/quantization-recall.html — standalone harness. Open in
  browser, reads recall@10 as text. Mulberry32-seeded for repro.

Also updates the phase tracker in docs/plan/rulake-inspired-features.md
to reflect Phase 1 wave 1 closed (1B + 1D shipped); wave 2 (1A + 1C)
remains open because both touch ruvectorBridge.js stubs and would race.

Validated via agent-browser:
- Determinism: quantize(v).packed equal byte-for-byte across calls.
- Hamming(v,v) == 0 on identity.
- Recall@10 = 0.9060 ≥ 0.9 gate on 100 synthetic brain vectors.
- Main app boots cleanly with no new console errors after adding the
  quantization/ directory (nothing imports it yet, so it's a no-op on
  the current code path).
F3 from docs/plan/rulake-inspired-features.md. Serializes the whole
brain/track/dynamics archive to a gzipped JSON bundle (.vvarchive.json.gz)
and rehydrates it deterministically via replay-of-insertions — no WASM
patch needed. Behind ?snapshots=1 while it bakes.

What this ships:
- archive/exporter.js — buildSnapshot({...}) → ArchiveSnapshot with
  xxHash32 witness (sync, click-handler safe). Also buildSnapshotAsync
  with crypto.subtle SHA-256 witness when an async path is acceptable.
- archive/importer.js — applySnapshot(snapshot, targets) replays into
  fresh VectorDB instances, rebuilds mirrors, re-hydrates the lineage
  DAG. Respects insertionOrder; falls back to row order if absent.
- archive/serialize.js — toBlob/fromBlob via CompressionStream('gzip')
  with plain-JSON fallback for older Safari. Magic header
  "VVARCHIVE v1 {gzip|json}" so truncated downloads fail loudly at import.
- ruvectorBridge.js — exportSnapshot/importSnapshot stubs replaced with
  real bodies. _insertionOrder: string[] now tracked alongside the
  mirrors — pushed by archiveBrain, hydrate, hydrateFromFixture; cleared
  by _debugReset and importSnapshot; consumed by rebuildIndicesFromMirror
  so setIndexKind preserves graph identity. setConsistencyMode/
  getIndexStats stubs unchanged (1C / 3A own those).
- uiPanels.js — flag-gated "📦 Export archive" / "📥 Import archive" row.
  Renders only when ?snapshots=1; default UI unchanged.
- main.js — after bridge.ready(), if ?snapshots=1 and a staged import is
  pending, runs it pre-sim. Fire-and-forget; logs counts.
- eli15/chapters/warm-restart.js — full chapter replaces coming-soon stub.
- tests/warm-restart-roundtrip.html — standalone harness. Seeds fixture,
  exports → toBlob → fromBlob → importSnapshot, asserts 5 claims.

Validated via agent-browser:
- Round-trip harness prints PASS 5/5: validateSnapshot ok, importSnapshot
  returns right counts (brains=8 tracks=1), brain count preserved
  (pre=8 post=8), top-1 id byte-identical (brain-7 → brain-7), id set
  byte-identical. gzip=true (CompressionStream path exercised, not JSON
  fallback).
- Default URL (no flag): main app boots cleanly, no new errors, Export/
  Import buttons absent.
- Flag URL (?snapshots=1): both buttons render in the training panel.

Not yet integrated with F5 dedup (import still re-inserts rather than
skipping duplicates) — that's a later integration slice. Not yet
integrated with F4 consistency (snapshot.consistency defaults to 'fresh').
F4 from docs/plan/rulake-inspired-features.md. Three query-time modes
controlling how recommendSeeds sees the archive; closes Phase 1 of the
roadmap (only Phase 3A's getIndexStats stub remains in ruvectorBridge).

What this ships:
- consistency/mode.js — state machine + TTL cache + frozen-id-set ref.
  Exports getMode/setMode, recordQuery/getCachedResult, freezeArchive/
  thawArchive, stats, _debugReset. TTL default 10 generations.
- consistency/worker-sync.js — computeFrozenSyncPayload() helper for the
  A/B baseline worker; postMessage plumbing is a follow-up.
- ruvectorBridge.js — setConsistencyMode stub body replaced. One branch
  added inside recommendSeeds that reads the mode:
    * fresh  → unchanged path (single no-op boolean per candidate when
      frozenIds is null — fresh-mode behaviour byte-identical to pre-1C)
    * eventual → cache hit returns memoized result, else fresh query +
      recordQuery
    * frozen → filter candidates to ids in the pinned Set
  Freeze mechanism is the cap-by-insertion-id shortcut (plan's rec): at
  freeze-entry, snapshot new Set(_brainMirror.keys()); new archiveBrain
  calls still insert (archive growth uninterrupted), their ids just
  aren't in the pinned set. Rejected the alternatives (VectorDB clone,
  parallel read-only index) as 2× memory for no spec benefit.
- uiPanels.js — new radio row (Fresh/Eventual/Frozen, default Fresh)
  with a 30-span tick strip that pulses on cacheMisses — visible
  evidence of when the archive is actually re-queried.
- style.css — .rv-consistency* classes including keyframe pulse.
- main.js — ?consistency=fresh|eventual|frozen URL flag, mirrors the
  ?hhnsw=1 pattern; polls for bridge readiness before applying.
- sim-worker.js — new case 'freeze' branch gated on importSnapshot
  availability; existing A/B setups without frozen-mode unchanged.
- eli15/chapters/consistency-modes.js — full chapter with the three
  modes, the A/B-race motivation for frozen, tick-strip semantics.
- tests/consistency-modes-smoke.html — 5-claim harness.

Also updates docs/plan/rulake-inspired-features.md to reflect Phase 1
complete; M1 milestone (F1+F3 demoable) checked off.

Validated via agent-browser:
- Smoke harness PASS 5/5: fresh bypasses cache (0 hits 0 misses),
  eventual 2nd call is a hit (sameRef=true), frozen excludes post-
  freeze brain, thaw restores visibility, no throws on valid modes.
- Default UI: Fresh mode, all 3 radios + tick strip render, app boots
  cleanly with no new console errors.
- ?consistency=frozen URL flag: bridge reports 'frozen', Frozen radio
  checked.

Phase 1 ordering discipline held: 4 subagents (1B+1D in parallel,
1A→1C sequentially) edited 15+ files, zero merge conflicts.
F2 from docs/plan/rulake-inspired-features.md. Query path can now
fan out to BOTH nearest-neighbour indexes simultaneously, union the
candidates by hash (F5), and GNN-rerank the union. Gated behind
?federation=1 or the UI toggle; off by default. Composes with Phase
1C consistency modes (frozen filter applies to the union; eventual
cache stores the final federated result, not per-shard).

What this ships:
- federation/fanout.js — fanOut(queryVec, k, shards) returns per-shard
  top-k'. Over-request formula kPrime(k, S) = k + ⌈√(k·ln S)⌉,
  clamped so S≤1 returns k (graceful single-shard degrade). Also a
  fanOutSync twin for tests.
- federation/rerank.js — unionByHash (dedup key is hash; tracks
  per-candidate shard provenance), selectTopK (picks final k by
  external score map; falls back to bestScore when unscored).
- federation/viewer.js — plain-DOM split-screen: Euclidean top-k' on
  the left, Hyperbolic top-k' on the right, unioned+reranked top-k in
  the middle, with the formula rendered live.
- ruvectorBridge.js — structural addition of _brainDB_hyperbolic as a
  shadow of _brainDB populated in lock-step across archiveBrain,
  hydrate, hydrateFromFixture, rebuildIndicesFromMirror, and
  importSnapshot. The existing setIndexKind swap still works — the
  primary index swaps, the shadow stays hyperbolic. New
  _recommendSeedsFederated helper handles the fanout branch; wired into
  recommendSeeds so the 1C consistency cache/frozen logic wraps both
  the single-index and federated paths. New exports:
  setFederationEnabled / isFederationEnabled / getFederationStats /
  setFederationCapturer.
- uiPanels.js — new "🌐 Federation" toggle + mount point for the
  viewer. Renders federation stats on every UI tick.
- main.js — ?federation=1 URL flag, mirrors the 1C pattern.
- eli15/chapters/federation.js — full chapter with the over-request
  formula worked example (k=10 → k'=13), dedup-via-F5 explanation,
  compose-with-1C-and-1D section.
- tests/federation-smoke.html — 7-claim harness.

Validated via agent-browser:
- Harness PASS 7/7. lastKPrime=7 matches k=5 + ⌈√(5·ln 2)⌉=7 exactly.
  shards=2 (both loaded). dedupeHits=4 (both shards surfaced 4
  overlapping brains and the hash union correctly collapsed them).
  Frozen-mode filter correctly applied to the union, not per-shard
  (post-freeze brain absent from federated result).
- Main app boots cleanly without any flag, no new console errors.
- Exactly one "not implemented" stub remains in ruvectorBridge.js
  (getIndexStats, Phase 3A/F7).

Not yet integrated with F1 quantization (per plan scope note) — that
would be a later integration slice. Not yet integrated with F6
cross-tab — that's Phase 2B, gated on 1A's snapshot format (which
shipped) but sequential with this commit on ruvectorBridge.js.
F6 from docs/plan/rulake-inspired-features.md. Closes Phase 2. When
two tabs on the same origin open with ?crosstab=1, a brain archived
in tab A arrives in tab B's archive within a microtask. No server,
no sockets — pure browser primitive. F5's content-hash IDs make the
protocol lockless by construction: two tabs computing the same brain
produce the same hash, dedup collapses the repeat automatically.

What this ships:
- crosstab/channel.js — BroadcastChannel('vectorvroom-archive') wrapper.
  Per-tab random senderId; hello/hello-ack/bye protocol for peer count;
  graceful beforeunload bye; stats() for diagnostics.
- crosstab/wire.js — single-brain payload codec. flat: Array<number>
  on the wire (not Float32Array) so two tabs on slightly different
  builds still interop. hash is sender-computed but receiver
  re-derives via dedup.
- ruvectorBridge.js — additive only. New state: _crosstabEnabled,
  _crosstabReceiving, _crosstabChannel, _crosstabSenderId. Broadcast
  hook in archiveBrain guarded by (_crosstabEnabled && !_crosstabReceiving).
  _onRemoteBrain wraps the local archiveBrain call in
  _crosstabReceiving=true/try/finally so the receive path is
  pure-one-way, never re-broadcasts. New exports setCrosstabEnabled /
  isCrosstabEnabled / getCrosstabStats / setCrosstabListeners /
  _onRemoteBrain.
- uiPanels.js — flag-gated "🔗 N peer(s)" pill; pulses on received
  brain via a CSS class toggle.
- style.css — .rv-crosstab-pill + rv-crosstab-pulse keyframes.
- main.js — ?crosstab=1 URL flag applier, mirrors 1C / 2A pattern.
- eli15/chapters/cross-tab-federation.js — full chapter replacing
  coming-soon stub: BroadcastChannel basics, why content-addressing
  makes it lockless, echo-prevention via _crosstabReceiving guard,
  worked two-tab example.
- tests/crosstab-smoke.html — 3-claim harness simulating two tabs
  with two channels in one page.

Also updates docs/plan/rulake-inspired-features.md: 2B closed, Phase 2
complete; milestone M2 ticked (Phase 1 merged); M3 in progress.

Validated via agent-browser:
- Harness PASS 3/3: A→B delivery within 500ms with full wire decode
  (flat=244, fitness=42.5, hashMatch=true); duplicate brain dedups to
  one canonical node with duplicateRatio=0.5 after 2 inserts;
  sender-echo suppression verified (0 self-echoes at A, 1 from B).
- ?crosstab=1 flag enables crosstab: isCrosstabEnabled()=true, pill
  rendered, stats show started=true with a per-tab senderId.
- Default boot (no flag) clean, no new console errors.

Echo-loop guard: _crosstabReceiving module-scope boolean flipped
around the receive-path archiveBrain call. Broadcast hook checks it.
Belt-and-braces with the senderId===self drop in channel.js (which
is redundant today since BroadcastChannel doesn't echo to sender,
but documented as defense-in-depth for future relay topologies).

Only Phase 3A's getIndexStats stub remains unimplemented in the
bridge. Phase 2 sequencing discipline held: 2A→2B sequentially,
both structural edits to ruvectorBridge.js, zero merge conflicts.
F7 from docs/plan/rulake-inspired-features.md. Per-stage timings for
every generation, rendered as a collapsible stacked-bar + numeric
table below the vector-memory panel. This commit fills the last
"not implemented" stub in ruvectorBridge.js — the whole bridge now
has zero unimplemented exports for the first time since Phase 0.

What this ships:
- observability/timings.js — ring-buffer histogram per stage name,
  window=20 generations. startStage/endStage with try/finally
  exception-safety, record for one-shot, snapshot returns per-stage
  {count, totalMs, avgMs, lastMs, p95Ms}.
- observability/panel.js — plain-DOM collapsible panel with rAF-gated
  10Hz auto-refresh. Stacked horizontal bar (div % widths) + details
  table. Inline CSS scoped to .rv-obs-panel.
- ruvectorBridge.js — getIndexStats() body replaced. Returns a live
  6-key snapshot: archive counts, index kind, federation/consistency/
  crosstab stats (each try/catch-wrapped for optional getters), and
  timings. Stage timers added around recommendSeeds hot-path stages
  (retrieve, federate, rerank, adapt, dynamics). setGeneration called
  inside archiveBrain. Federation branch timer wraps the whole fanout
  to avoid double-counting its inner rerank.
- uiPanels.js — dynamic-import mount point below the existing panel,
  anchored by "=== Phase 3A observability mount point ===" so Phase 3C
  can add its own section without conflict.
- eli15/chapters/where-the-time-goes.js — full chapter replacing the
  Phase 0 coming-soon stub. Covers each stage, how to read p95, and
  the "surprise big number" workflow.
- tests/observability-smoke.html — 5-claim harness.

Validated via agent-browser:
- getIndexStats returns the full 6-key structure.
- Observability panel renders in the UI (default on; telemetry, no
  behaviour change).
- Harness 5/5 PASS including the ring-buffer moving-average check:
  25 records into window=20 yields count=20 avgMs=15.5 = (6+25)/2,
  i.e. only the trailing 20 samples are reflected.
- Main app boots cleanly, no new console errors.

Load-bearing claim confirmed: `grep 'not implemented' ruvectorBridge.js`
returns zero hits. The Phase 0 pre-partition is fully consumed.
… cross-links)

3B from docs/plan/rulake-inspired-features.md. No chapter body edits;
registry hygiene and pedagogical ordering only.

What changed:
- eli15/index.js — registry re-ordered into four pedagogical bands:
  Foundations → Vector memory basics → Lineage & reasoning → Geometry
  choice → RuLake-inspired extensions. Inside the extensions band:
  content-addressing first (hash primitive), then the features that
  compose with it (quantization, warm-restart, consistency, federation,
  cross-tab, observability last). Added `related: [...]` cross-link
  fields to the 7 roadmap chapter entries. comingSoon: true flags
  removed from every chapter whose file body no longer has it —
  including where-the-time-goes (reconciled post-3A land).
- eli15/tour.js — STEPS array appended with the 7 RuLake-inspired
  extensions as conceptual-only steps (anchor: null) matching the
  registry order. Tour is driven by explicit STEPS, not
  Object.keys(REGISTRY).

Suggested chapter order from the plan was followed with two
documented deviations:
1. track-similarity before ema-reranker (reads more naturally once the
   learner has seen a retrieval happen).
2. content-addressing first inside the extensions band (F5 hash is
   the primitive every subsequent feature depends on).

Validated:
- Main app boots cleanly.
- ELI15 drawer opens; chapter list reflects the new order (head:
  what-is-this-project → sensors → ...; tail: federation → cross-tab-
  federation → where-the-time-goes).
- Opening where-the-time-goes renders real content, no "Coming soon"
  artefact.
…admap complete)

3C from docs/plan/rulake-inspired-features.md. Closes Phase 3 and the
whole RuLake-inspired roadmap. Archive bundles from Phase 1A can now
travel between users via a shareable URL — we don't host anything,
the user supplies the URL (gist / S3 / IPFS / anywhere with CORS).

What this ships:
- share/url.js — fetchArchive(url) pipes through serialize.fromBlob +
  validateSnapshot, throws with readable errors on CORS/404/validate.
  buildShareUrl(archiveUrl) constructs the shareable link preserving
  other flags (?consistency=, ?federation=...), stripping any pre-
  existing snapshots/archive params so the link is deterministic.
- share/gallery.js — curated community-archive list. Ships with
  exactly ONE placeholder entry (url: 'about:blank') per the
  external-scope rule: real URLs require explicit user approval
  before publication. A code comment lists candidate entries queued
  for review but not activated.
- uiPanels.js — new share panel gated by ?snapshots=1, anchored with
  "=== Phase 3C share panel ===" so future edits can find its spot
  without fighting 3A's observability block. Copy-shareable-link
  button (with alert fallback for insecure contexts where
  navigator.clipboard is unavailable), Import-from-URL input+button,
  gallery mount.
- main.js — ?archive=<url> URL flag. Requires ?snapshots=1 to be set
  alongside (same gate as Phase 1A). Auto-fetches + imports at boot.
- eli15/chapters/warm-restart.js — appended "Sharing a museum over
  the internet" section at the end of the existing body. No rewrite
  of existing content (per plan: 3C extends 1A's chapter rather than
  adding a new one).
- tests/share-smoke.html — 4-claim harness mocks window.fetch against
  an in-memory buildSnapshot+toBlob fixture.

Also updates docs/plan/rulake-inspired-features.md: every Phase row
✅, all four milestones M1-M4 closed, decisions log records the
final phase narrative. Focus line set to "Roadmap complete".

Validated via agent-browser:
- Harness PASS 4/4: fetchArchive + validateSnapshot round-trip; share
  URL exactly `?snapshots=1&archive=<url-encoded>`; gallery click
  invokes onImport with 'about:blank'; GALLERY.length === 1.
- ?snapshots=1 flag renders the share panel with both buttons and the
  one placeholder gallery entry. Default (no flag): UI unchanged.
- Main app boots cleanly with no new console errors.

External-scope discipline held: no real community archive URLs were
added in this commit. The gallery is a single about:blank sentinel
that fails fetch loudly by design until a vetted URL lands.

Roadmap stats: 10 commits today (Phase 0 foundations + 9 feature
commits), ~9,400 lines of feature code, 8 browser-live test harnesses,
8 ELI15 chapters rewritten from Phase 0 stubs, 4 subagent swarm waves
(1B+1D parallel, 1A seq, 1C seq, 2A seq, 2B seq, 3A+3B parallel, 3C
seq). Zero merge conflicts, zero rollbacks, zero "not implemented"
stubs remaining in ruvectorBridge.js.
@shaal shaal merged commit ee2dd34 into main Apr 24, 2026
3 of 5 checks passed
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 24, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7fa6c026-4780-4f8b-a927-7eec5706a535

📥 Commits

Reviewing files that changed from the base of the PR and between 4c2527b and 93c9dc4.

📒 Files selected for processing (47)
  • AI-Car-Racer/archive/dedup.js
  • AI-Car-Racer/archive/exporter.js
  • AI-Car-Racer/archive/hash.js
  • AI-Car-Racer/archive/importer.js
  • AI-Car-Racer/archive/serialize.js
  • AI-Car-Racer/archive/snapshot.js
  • AI-Car-Racer/brainExport.js
  • AI-Car-Racer/consistency/mode.js
  • AI-Car-Racer/consistency/worker-sync.js
  • AI-Car-Racer/crosstab/channel.js
  • AI-Car-Racer/crosstab/wire.js
  • AI-Car-Racer/eli15/chapters/consistency-modes.js
  • AI-Car-Racer/eli15/chapters/content-addressing.js
  • AI-Car-Racer/eli15/chapters/cross-tab-federation.js
  • AI-Car-Racer/eli15/chapters/federation.js
  • AI-Car-Racer/eli15/chapters/quantization.js
  • AI-Car-Racer/eli15/chapters/warm-restart.js
  • AI-Car-Racer/eli15/chapters/where-the-time-goes.js
  • AI-Car-Racer/eli15/index.js
  • AI-Car-Racer/eli15/tour.js
  • AI-Car-Racer/federation/fanout.js
  • AI-Car-Racer/federation/rerank.js
  • AI-Car-Racer/federation/viewer.js
  • AI-Car-Racer/lineage/dag.js
  • AI-Car-Racer/lineage/viewer.js
  • AI-Car-Racer/main.js
  • AI-Car-Racer/observability/panel.js
  • AI-Car-Racer/observability/timings.js
  • AI-Car-Racer/quantization/hadamard.js
  • AI-Car-Racer/quantization/index.js
  • AI-Car-Racer/quantization/rabitq.js
  • AI-Car-Racer/quantization/viewer.js
  • AI-Car-Racer/ruvectorBridge.js
  • AI-Car-Racer/share/gallery.js
  • AI-Car-Racer/share/url.js
  • AI-Car-Racer/sim-worker.js
  • AI-Car-Racer/style.css
  • AI-Car-Racer/uiPanels.js
  • docs/plan/rulake-inspired-features.md
  • tests/consistency-modes-smoke.html
  • tests/crosstab-smoke.html
  • tests/dedup-smoke.html
  • tests/federation-smoke.html
  • tests/observability-smoke.html
  • tests/quantization-recall.html
  • tests/share-smoke.html
  • tests/warm-restart-roundtrip.html

📝 Walkthrough

Walkthrough

This pull request introduces a comprehensive Phase 2A-3B feature set: archive snapshots for warm restart with gzip serialization; federated vector search with union/reranking; 1-bit quantization via Hadamard rotation; consistency modes (fresh, eventual, frozen) with TTL caching; cross-tab synchronization via BroadcastChannel; content-addressed brain deduplication; observability/timing instrumentation; and sharing with community gallery support. Includes six new Eli15 chapters, multiple smoke tests, and extensive documentation of the RuLake-inspired roadmap.

Changes

Cohort / File(s) Summary
Archive Core
archive/snapshot.js, archive/hash.js, archive/dedup.js
Schema definition, xxHash32 hashing, and session-wide deduplication state management with duplicate counting.
Archive Serialization
archive/serialize.js, archive/exporter.js, archive/importer.js
Snapshot blob encoding/decoding with gzip+JSON support; deterministic snapshot building with xxHash32/SHA256 witness; and replay-mode hydration with vector reconstruction.
Consistency Management
consistency/mode.js, consistency/worker-sync.js
Stateful mode API (fresh/eventual/frozen) with per-query TTL caching, frozen snapshot pinning, and worker freeze-sync payloads.
Federation (Phase 2A)
federation/fanout.js, federation/rerank.js, federation/viewer.js
Per-shard over-requesting computation; union dedup by content hash; top-k reranking; and DOM viewer with split-screen metrics/stats.
Quantization (Phase 2B)
quantization/hadamard.js, quantization/rabitq.js, quantization/index.js, quantization/viewer.js
FWHT-based Hadamard rotation; 1-bit sign-packing with float residual; Hamming and cosine estimators; public re-export surface; and heatmap/scatter/metrics viewer.
Cross-tab Sync
crosstab/channel.js, crosstab/wire.js
BroadcastChannel wrapper with peer tracking, sender echo suppression, and wire format serialization/deserialization for broadcast payloads.
Observability
observability/timings.js, observability/panel.js
In-memory ring-buffer histograms per stage; p95 computation; and DOM panel with live stats polling and "learn more" ELI15 integration.
Sharing & Community
share/url.js, share/gallery.js
Archive fetch/validate from URL; shareable link builder; and minimal gallery panel with placeholder-only entry enforcement.
Core Integration
ruvectorBridge.js, brainExport.js, lineage/dag.js
Federated retrieval with hyperbolic shadow index; cross-tab receive via remote brain reinsertion; consistency filtering; hash-keyed dedup DAG tracking; and duplicate-count node metadata.
UI & Main
uiPanels.js, main.js, style.css
Consistency mode radio controls with tick-strip animation; federation toggle + viewer mounting; archive export/import; crosstab pill; observability auto-mount; share panel with gallery; and new CSS animations/classes.
Worker & Simulator
sim-worker.js
Freeze message handling to conditionally import snapshot and switch to frozen consistency mode.
Lineage Viewer
lineage/viewer.js
Per-node visual sighting badge (×N) and tooltip duplicate suffix for absorbed duplicates.
Documentation & Roadmap
eli15/chapters/{consistency-modes,content-addressing,cross-tab-federation,federation,quantization,warm-restart,where-the-time-goes}.js, eli15/index.js, eli15/tour.js, docs/plan/rulake-inspired-features.md
Seven new chapters with HTML bodies; chapter registry reordering and new lazy-loader wiring; seven new interstitial tour steps; and comprehensive multi-phase roadmap with task specs, decisions log, and confidence gates.
Smoke Tests
tests/{consistency-modes,crosstab,dedup,federation,observability,quantization-recall,share,warm-restart-roundtrip}-smoke.html
Eight standalone browser test harnesses covering dedup idempotence, consistency stats, federation k-prime/union/rerank, cross-tab peer behavior, snapshot round-trip, quantization recall@10, and share URL/gallery flows.

Sequence Diagram(s)

sequenceDiagram
    participant Client as UI/Main
    participant Bridge as ruvectorBridge
    participant Euclidean as Euclidean<br/>Index
    participant Hyperbolic as Hyperbolic<br/>Index
    participant Union as Union &<br/>Dedup
    participant Rerank as Rerank &<br/>Score
    participant Cache as Consistency<br/>Cache

    Client->>Cache: getConsistencyMode()
    alt eventual mode + cache hit
        Cache-->>Client: return cached result
    else fresh or eventual miss
        Client->>Bridge: recommendSeeds(queryVec, k)
        alt federation enabled
            Bridge->>Euclidean: search(queryVec, k')
            Bridge->>Hyperbolic: search(queryVec, k')
            Euclidean-->>Union: results
            Hyperbolic-->>Union: results
            Union->>Union: dedup by content hash
            Union->>Rerank: union candidates
            Rerank->>Rerank: score by GNN/EMA<br/>apply frozen filter
            Rerank-->>Bridge: top-k results
        else federation disabled
            Bridge->>Euclidean: search(queryVec, k)
            Euclidean-->>Bridge: top-k results
        end
        Bridge-->>Cache: record result
        Cache->>Client: return result
    end
Loading
sequenceDiagram
    participant UI as UI Panel
    participant Export as exportSnapshot()<br/>(ruvectorBridge)
    participant Build as buildSnapshot()
    participant Serialize as toBlob()
    participant File as File/Network
    participant Deserialize as fromBlob()
    participant Validate as validateSnapshot()
    participant Import as importSnapshot()
    participant Bridge as ruvectorBridge

    UI->>Export: click export
    Export->>Build: construct snapshot<br/>with mirrors & HNSW state
    Build->>Build: compute xxHash32/SHA256<br/>witness
    Build-->>Export: snapshot object
    Export->>Serialize: toBlob(snapshot)
    Serialize->>Serialize: JSON + gzip<br/>(if available)
    Serialize-->>UI: Blob
    UI->>File: save or share URL
    
    alt Import from file or URL
        File->>Deserialize: fromBlob(blob)
        Deserialize->>Deserialize: detect codec,<br/>decompress
        Deserialize-->>Validate: snapshot object
        Validate->>Validate: verify schema version<br/>& witness
        Validate-->>UI: OK
        UI->>Import: importSnapshot(snapshot)
        Import->>Bridge: clear mirrors,<br/>insert tracks/dynamics/brains<br/>in insertion order
        Import->>Bridge: load observations
        Bridge->>Bridge: rebuild HNSW<br/>from insertionOrder
        Bridge-->>UI: counts & success
    end
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes


Poem

🐰 With archives now frozen and hashes so snappy,
Quantized brains hop across tabs, oh so happy!
Through federation's fanout, dedup keeps things pure,
While consistency modes let the cache endure.
VectorVroom speeds onward, content-addressed and blessed! 🚀

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rulake-roadmap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@shaal shaal deleted the rulake-roadmap branch April 24, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant