Skip to content

perf: setEmbedding invalidates the whole vector cache every batch → full O(N) matrix reload per recall #86

Description

@saucam

Summary

setEmbedding invalidates the workspace's memoized vector matrix on nearly every embed batch. Since batches fire continuously during active work, the next recall() re-reads every embedding BLOB from SQLite and re-allocates the full matrix before the cosine loop even starts — O(N) I/O + allocation per query that should be O(1).

Location

  • src/daemon/memory/store.ts:241 / :254 (cache invalidation on write)
  • rebuild at src/daemon/memory/store.ts:742-762; uint8ToFloat32 allocates twice per row at :845-851
  • invoked from src/daemon/memory/engine.ts:338-345
  • query-time brute force + full sort at src/daemon/memory/engine.ts:159-171

Failure scenario

50k-episode workspace (384-dim ≈ 77 MB of vectors): every recall during active work pays a full-table BLOB scan + ~150 MB of transient allocation before scoring, hundreds of ms per query plus GC churn, all on the shared daemon event loop. Worse while embedding is ongoing (the common case).

Fix

The id and vector are both known at setEmbedding time — append the new row to the cached matrix incrementally instead of clearing it. (Related but separate: #71 covers replacing the brute-force cosine with an ANN backend.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    auditSurfaced in a code auditperfPerformance and scalability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions