v0.96.0

Added

• A keyword index rebuild can be watched and waited for. KeywordIndexStatistics.Rebuild (from
  GetStatisticsAsync()) reports the re-tokenization an index started on opening under a different analyzer or field
  set — Change, Completed / Total chunks, Status (Running, Interrupted, Failed) — and
  IKeywordSearchService.WaitForMaintenanceAsync() completes when it has finished (default implementation: completes at
  once, so other implementations keep compiling).
• An interrupted rebuild continues where it stopped. Each batch commits a cursor with its rows; a host stopped
  part-way (deploy, restart, health-check kill) continues after the last committed batch on its next open. A cursor
  recorded toward a different analyzer or field set is dropped. Processes sharing one PostgreSQL index take turns batch
  by batch instead of each rewriting it.

Changed

• Breaking: opening a relational keyword index no longer waits for the rebuild a changed analyzer or field set
  needs. EnsureSchemaAsync and the first keyword call return once the schema is ready; the rebuild runs in the
  background, and searches meanwhile see the index as it stands (partial keyword recall until it completes). Writes
  are not blocked. A rebuild batch reads its chunks after locking them, so a chunk rewritten or deleted during the
  rebuild ends as the writer left it. Migration: if your host must not serve partial keyword recall, await
  WaitForMaintenanceAsync() on the keyword service before serving.
• Keyword-index writes cost a few statements per batch instead of one per posting. Indexing, re-indexing, document
  replacement and the definition rebuild now delete a batch's postings in one statement per relation and write its
  chunks, postings, field postings and filter rows as set inserts (PostgreSQL: one unnest insert per relation;
  SQLite: multi-row VALUES). Terms that already have a row are looked up instead of re-inserted, and only the term
  rows whose document frequency moves are locked and checked for cleanup. Measured on PostgreSQL in Docker, rebuilding
  2,000 stored chunks whose terms did not change went from 220 s to 17 s.
• The plain index idx_bm25_terms_term is dropped on first open: bm25_terms.term is UNIQUE, which already indexes
  it, and the second index doubled the cost of every term insert and document-frequency update.
• Breaking (custom keyword backends only): RelationalKeywordSearchService no longer has the abstract
  UpsertChunkSql, UpsertPostingSql and UpsertFieldPostingSql; rows are written through the new protected virtual
  InsertRowsAsync(connection, table, columns, values, conflictClause, cancellationToken), whose default writes
  multi-row VALUES. Migration: delete those three overrides from a subclass; override InsertRowsAsync only if your
  database has a faster array-valued insert.

Fixed

• A keyword index written before 0.93.0 under the analyzer you still register is no longer re-tokenized on first
  open. Such an index records no analyzer; the first open now re-analyzes an evenly spaced sample of up to 1,024 stored
  chunks and compares the result with their stored body and field postings. When every sampled chunk matches, the
  analyzer is recorded as it stands (one information line, no rewrite); when one differs, the index is re-tokenized
  once as before, and the warning names the differing chunk.
• Spellings an analyzer emits in different case now count as one term. A custom analyzer that keeps case (Index,
  index) wrote one posting whose frequency was the last spelling's count instead of their sum; the frequency is now
  the sum, so BM25 ranks such a chunk as it ranks the lower-cased text.

Dependencies

• Re-pinned sibling package(s) FileFlux 0.63.0 -> 0.63.1, LMSupply.Embedder 0.121.0 -> 0.122.0, LMSupply.Generator 0.121.0 -> 0.122.0, LMSupply.Reranker 0.121.0 -> 0.122.0. FluxIndex.Providers.LMSupply 0.96.0 is built against LMSupply 0.122.0, which renamed the embedder's ModelInfo to EmbedderModelInfo: move LMSupply.* and FluxIndex.Providers.LMSupply together (0.95.0 with LMSupply 0.122.0 fails at run time).