Skip to content

v0.24.0

Choose a tag to compare

@github-actions github-actions released this 17 Jul 22:47

slater 0.24.0 — dot-product (MIPS) vector search + a security & correctness hardening pass

Makes dot-product (inner-product / MIPS) a first-class vector metric and folds in a
whole-codebase review remediation. Builds on the v0.23.0 FreshDiskANN writable vector
ladder. No on-disk format-version change — existing generations open unchanged.

Dot-product (MIPS) vector search — now properly supported (HIK-137)
Dot-product kNN previously recalled only ~0.32–0.50: the graph reached inner-product
neighbours via a norm-augmentation reduction that navigates poorly under wide norm
spreads. 0.24.0 adds an IP-native navigator — inner-product closeness, top-R-by-IP
neighbour selection, highest-norm entry, and an IP-trained PQ codebook — across the whole
write ladder: delta, sealed segment, consolidate, merge and delete. On the measurement
fixture recall@10 rises from ~0.40 to ~0.99–1.0 and holds rung-to-rung, verified against an
independent brute-force inner-product ground truth. The navigator is an additive, optional
discriminator: cosine and L2 navigation is unchanged, and a forged inner-product navigation
on a cosine/L2 index is refused rather than mis-served.

Security & correctness hardening (whole-codebase review)

  • Session state no longer outlives the identity it belonged to — LOGOFF / failed re-LOGON
    could leave one connection's buffered rows and open-transaction graph readable by the next
    user on that connection, bypassing the read ACL. (HIK-123)
  • A label removal is a scope change, not an embedding delete — consolidation no longer
    permanently destroys a de-labelled node's vector. (HIK-122)
  • Disk cache is bounded across restarts (adopts prior-run files, LRU-evictable) and its
    write-behind queue is byte-bounded — closing an unbounded-disk path and an unaccounted-RSS
    path. (HIK-124, HIK-125)
  • On-disk widths, offset tables and PQ code bytes are validated where they enter memory — a
    corrupt or forged block now errors instead of panicking or silently returning wrong
    results. (HIK-126, HIK-128, HIK-132, HIK-133)
  • Non-finite embedding components are rejected at ingest and query; out-of-range duration
    components are rejected instead of saturating to a wrong value; WAL tail and commit-marker
    handling corrected. (HIK-134, HIK-127, HIK-130, HIK-135)
  • User range indexes are retained on the pk property during build. (HIK-129)

Tooling & tests

  • A committed, reproducible performance suite for the vector write ladder, with recall scored
    against an independent brute-force ground truth (docs/PERF-REPORT.md). (HIK-120, HIK-121)
  • CI now compiles and runs the s3/gcs feature-gated test code that the default build never
    touched — the blind spot where the disk-cache bugs had survived. (HIK-138)

Upgrade notes
No format-version change; v0.23.0 generations open as-is. Dot-product graphs opt into the new
IP-native navigation; cosine/L2 behaviour is unchanged.

Full Changelog: v0.23.0...v0.24.0