v0.24.0
slater 0.24.0 — dot-product (MIPS) vector search + a security & correctness hardening pass
Makes dot-product (inner-product / MIPS) a first-class vector metric and folds in a
whole-codebase review remediation. Builds on the v0.23.0 FreshDiskANN writable vector
ladder. No on-disk format-version change — existing generations open unchanged.
Dot-product (MIPS) vector search — now properly supported (HIK-137)
Dot-product kNN previously recalled only ~0.32–0.50: the graph reached inner-product
neighbours via a norm-augmentation reduction that navigates poorly under wide norm
spreads. 0.24.0 adds an IP-native navigator — inner-product closeness, top-R-by-IP
neighbour selection, highest-norm entry, and an IP-trained PQ codebook — across the whole
write ladder: delta, sealed segment, consolidate, merge and delete. On the measurement
fixture recall@10 rises from ~0.40 to ~0.99–1.0 and holds rung-to-rung, verified against an
independent brute-force inner-product ground truth. The navigator is an additive, optional
discriminator: cosine and L2 navigation is unchanged, and a forged inner-product navigation
on a cosine/L2 index is refused rather than mis-served.
Security & correctness hardening (whole-codebase review)
- Session state no longer outlives the identity it belonged to — LOGOFF / failed re-LOGON
could leave one connection's buffered rows and open-transaction graph readable by the next
user on that connection, bypassing the read ACL. (HIK-123) - A label removal is a scope change, not an embedding delete — consolidation no longer
permanently destroys a de-labelled node's vector. (HIK-122) - Disk cache is bounded across restarts (adopts prior-run files, LRU-evictable) and its
write-behind queue is byte-bounded — closing an unbounded-disk path and an unaccounted-RSS
path. (HIK-124, HIK-125) - On-disk widths, offset tables and PQ code bytes are validated where they enter memory — a
corrupt or forged block now errors instead of panicking or silently returning wrong
results. (HIK-126, HIK-128, HIK-132, HIK-133) - Non-finite embedding components are rejected at ingest and query; out-of-range duration
components are rejected instead of saturating to a wrong value; WAL tail and commit-marker
handling corrected. (HIK-134, HIK-127, HIK-130, HIK-135) - User range indexes are retained on the pk property during build. (HIK-129)
Tooling & tests
- A committed, reproducible performance suite for the vector write ladder, with recall scored
against an independent brute-force ground truth (docs/PERF-REPORT.md). (HIK-120, HIK-121) - CI now compiles and runs the s3/gcs feature-gated test code that the default build never
touched — the blind spot where the disk-cache bugs had survived. (HIK-138)
Upgrade notes
No format-version change; v0.23.0 generations open as-is. Dot-product graphs opt into the new
IP-native navigation; cosine/L2 behaviour is unchanged.
Full Changelog: v0.23.0...v0.24.0