v0.9.4 — cold-tier recall decompression fix
v0.9.4 — hotfix
Fixes #62: cold-tier recall scores were silently wrong — garbage similarities always, and a chance NaN that poisoned the response-level confidence.
Root cause (two compounding defects)
archive()stores cold-tier embeddings zstd-compressed in the same column hot rows use for raw f32 — and the recall read path never decompressed (decompress_embeddinghad exactly one call site:hydrate). Every recall scoring path that touched a cold record reinterpreted compressed bytes as floats. The cold tier has been silently unscoreable in recall since compression was introduced; #60''s panic fix is what made it visible as a wrong number instead of a crash.consolidate::cosine_similarity— the function recall actually scores through — still carried the pre-#60 NaN-blind guard (== 0.0).
Fix
decompress_if_cold()(zstd magic check) after decrypt in both durable embedding readers, mirroringhydrate(). Previously-affected records self-heal on the next read — no per-record hydrate ritual needed.- The #60 guard ported into
cosine_similarity(!(norm > 0.0)): degenerate vectors degrade to finite 0.0 similarity, never NaN. - Defense: the index rebuild scan also decompresses cold-format blobs if tier drift ever routes one there.
Credit: reported by @blue-mtn-dog with line-level root-cause analysis, verified exact on every claim — their second such report.
Published to PyPI (pip install -U yantrikdb) and crates.io.