Summary
Batch the per-trigram hashing in embed_text — it dominates bundle parse time.
Measured impact (spike #48, PR #84, B3)
embed_text is 37–78% of parse_bundle wall time across bundle trees (100–2500 bundles). It issues one hashlib.md5 call per character trigram (src/hotmem/embed.py:44). A native markdown/parse helper would optimize the wrong 20%; the lever is here.
Proposal
Batch trigram hashing (single hashlib update over a packed buffer, or a cheaper rolling hash), keeping hotmem-hash-v1 embedding semantics bit-compatible — otherwise version the embedding model string.
Acceptance criteria
hotmem-hash-v1 vectors unchanged (or model string versioned + migration noted).
- Measurable parse_bundle speedup on the spike's bundle trees (fixtures reusable).
- No behavior change for callers.
Context
Spike recommendation follow-up #5: bench/native_spike/README.md. Related: #48, #49.
Summary
Batch the per-trigram hashing in
embed_text— it dominates bundle parse time.Measured impact (spike #48, PR #84, B3)
embed_textis 37–78% ofparse_bundlewall time across bundle trees (100–2500 bundles). It issues onehashlib.md5call per character trigram (src/hotmem/embed.py:44). A native markdown/parse helper would optimize the wrong 20%; the lever is here.Proposal
Batch trigram hashing (single
hashlibupdate over a packed buffer, or a cheaper rolling hash), keepinghotmem-hash-v1embedding semantics bit-compatible — otherwise version the embedding model string.Acceptance criteria
hotmem-hash-v1vectors unchanged (or model string versioned + migration noted).Context
Spike recommendation follow-up #5:
bench/native_spike/README.md. Related: #48, #49.