perf: ANN index backend for vector search — replace brute-force cosine with usearch/hnswlib behind Embedder interface

[saucam]

Summary

Replace the brute-force cosine scan in store.ts's loadVectorMatrix() with an Approximate Nearest Neighbor (ANN) index (usearch or hnswlib) behind the existing Embedder-style interface. The current O(N) scan is correct and fast for small workspaces but hits memory and latency walls at scale.

Current behavior

```typescript
// store.ts — loadVectorMatrix()
// Loads ALL embeddings for a workspace into a Float32Array matrix
// engine.ts — recall()
const { ids, vectors } = this.#store.loadVectorMatrix(q.workspaceId);
for (let i = 0; i < vectors.length; i++) {
// brute-force cosine against every stored vector
for (let j = 0; j < queryVector.length; j++) sum += v[j]! * queryVector[j]!;
scored.push({ id: ids[i]!, score: sum });
}
```

The comment in store.ts is already honest about the scaling ceiling:

"Fine up to ~100k episodes on a laptop; swap to a real ANN index behind the same interface if we outgrow it."

Scale analysis

Episodes	Matrix size (384-dim float32)	Scan time (est.)
10K	15 MB	<5ms
50K	77 MB	~20ms
200K	307 MB	~80ms
500K	768 MB	OOM risk

At 50K+ episodes (a power user with months of sessions), the matrix load itself becomes the bottleneck, and at 200K+ it's a memory problem.

Proposed approach

VectorIndex interface

Extract the vector search responsibility into a pluggable interface alongside Embedder:

```typescript
// src/daemon/memory/vector-index.ts
export interface VectorIndex {
readonly size: number;
/** Add/update a vector for an episode id. /
upsert(id: string, vector: Float32Array): void;
/* Remove by id. /
remove(id: string): void;
/* Find top-K nearest neighbors to queryVector. Returns [id, score] pairs. /
search(queryVector: Float32Array, k: number): Array<{ id: string; score: number }>;
/* Persist index to disk (if backend supports it). /
save(path: string): Promise;
/* Load from disk. */
load(path: string): Promise;
}
```

Implementations

BruteForceVectorIndex (current behavior, default):

• In-memory Float32Array matrix
• Exact cosine, O(N) per query
• Zero deps — keep as default + fallback

UsearchVectorIndex (opt-in, production scale):

• usearch npm package — WASM-first, Bun-compatible, no native rebuild
• HNSW graph index, O(log N) search
• Persists to ~/.codeoid/vectors/<workspaceId>.usearch
• Fall back to BruteForce if usearch init fails

```typescript
import { Index } from 'usearch';

export class UsearchVectorIndex implements VectorIndex {
#index: Index;
constructor(dimensions: number) {
this.#index = new Index({ metric: 'cos', dimensions });
}
upsert(id: string, vector: Float32Array) {
this.#index.add(hashId(id), vector);
}
search(queryVector: Float32Array, k: number) {
const result = this.#index.search(queryVector, k);
return result.keys.map((key, i) => ({ id: unhashId(key), score: result.distances[i] }));
}
// ...
}
```

Config

```json
{
"memory": {
"vectorBackend": "bruteforce" // "bruteforce" (default) | "usearch"
}
}
```

Migration

On first startup with vectorBackend: "usearch", the engine reads all existing embeddings from SQLite and populates the usearch index. Progress logged. Subsequent startups load the persisted index file (~5× smaller than loading all raw Float32Arrays).

Why usearch over hnswlib

• usearch has a pure WASM npm build — no native compilation, works with Bun out of the box
• hnswlib-node requires native rebuild (problematic with Bun's Node compat layer)
• usearch supports cosine metric natively (BGE vectors are L2-normalized, cosine = dot product)
• usearch actively maintained; MIT licensed

Files

• New src/daemon/memory/vector-index.ts — VectorIndex interface + BruteForceVectorIndex + UsearchVectorIndex
• src/daemon/memory/store.ts — remove loadVectorMatrix(), replace with VectorIndex integration
• src/daemon/memory/engine.ts — accept VectorIndex in MemoryEngineOptions, use in recall()
• src/daemon/memory/index.ts — factory: pick backend from config, init index on startup
• src/config.ts — memory.vectorBackend config option

Acceptance criteria

• VectorIndex interface extracted and documented
• BruteForceVectorIndex passes all existing recall tests (behavior-identical to current)
• UsearchVectorIndex implemented, behind memory.vectorBackend: "usearch" config
• Migration path: existing SQLite embeddings → usearch index on first run
• Brute force remains default — zero behavior change for existing users
• usearch listed as optional dep — not required if brute force is used
• Benchmark: usearch recall latency at 50K episodes vs brute-force (target: <5ms)

Related

• src/daemon/memory/store.ts — loadVectorMatrix()
• src/daemon/memory/engine.ts — recall() vector branch
• Issue feat: file co-occurrence graph — 1-hop neighbor expansion after initial recall to surface related episodes #69 (file co-occurrence graph — uses same episode metadata, separate concern)
• Existing perf issue: perf: cache MemoryEngine vector matrix instead of reloading per recall — this supersedes that issue once usearch is in place