6.3.0

Search-correctness and data-integrity fixes. No API signature changes, but BM25 scores change and one previously-accepted input is now rejected — see Upgrade notes.

Fixed

BM25 text search returned no results on small corpora. The IDF used the textbook Okapi form, log((N - df + 0.5) / (df + 0.5)), which is exactly zero when a term appears in half the documents and negative above that. The score > 0 filter then discarded those documents entirely. In practice a one-document index never matched its own text, a term unique to one document in a two-document index scored exactly 0 and was dropped, and any term common to most of the corpus was discarded. In mixed queries a negative-IDF term actively ranked matching documents below non-matching ones. Now uses the Lucene form, log(1 + (N - df + 0.5) / (df + 0.5)), which is strictly positive for every df and monotonically decreasing, so relative ranking is preserved.

Documents could disappear from the BM25 index permanently. A rebuild suspends at loadDocuments(), so a concurrent indexDocument both missed the already-fixed snapshot and had its dirty mark cleared when the rebuild completed — no later search ever rebuilt, and the document stayed unsearchable for the process lifetime. Two overlapping rebuilds could also resume out of order, letting the older snapshot clobber a newer index. Both are now guarded by a mutation generation. unloadIndex() additionally detaches the index before awaiting unload(), since a search interleaving there previously observed a non-nil but emptied index and returned nothing.

FileStorageProvider's cache could go stale. A full load overlapping a save or delete installed its stale snapshot as the authoritative cache, dropping a just-saved document or resurrecting a just-deleted one — and because the cache was marked complete, later reads never went back to disk.

A partially failed batch save destroyed data. addDocuments called saveDocuments outside the do/catch guarding the indexing loop. Since writes run concurrently and successful ones are kept, a failed call could permanently destroy the content of any document its partial writes overwrote — while reporting failure to the caller — and leave orphaned documents on disk for IDs the caller was told were never added. The save now runs inside the rollback scope.

Duplicate IDs within a single batch collapsed into one document on disk while addDocuments reported both as added, with the surviving text decided by write ordering.

concurrentMap, concurrentForEach, and orderedConcurrentMap processed nothing at all when given a non-positive maxConcurrency, returning an empty array or silently skipping every side effect. Now clamped to 1.

Hardening: overflow guards on numResults * 2 and numResults * candidateMultiplier; TopKSelector no longer eagerly reserves Int.max capacity; memory-strategy parameters are clamped in VectorSearchEngine, which can be constructed directly and bypass VecturaConfig validation (a batchSize of 0 previously trapped in stride(by:)). Removed a duplicate Array.chunked(into:) overload that shadowed the size-guarded Collection version.

Upgrade notes

• BM25 and hybrid scores change. Absolute scores are higher and no longer clamp to zero for common terms, so a threshold tuned against 6.2.0 may now admit more results. Re-check thresholds if you tuned them empirically. Relative ranking is unchanged.
• addDocuments(texts:ids:) now throws VecturaError.invalidInput when ids contains duplicates. Previously it silently produced one document while reporting two as added.

Why minor, not patch

These are all bug fixes, but a patch release signals "no behavior change," which would be misleading: search scores move and a previously-accepted input now throws.

Full changelog: 6.2.0...6.3.0