Skip to content

v0.5.0: two-phase compute/store indexing, per-chunk prepare, FilterByRatio cost floor

Latest

Choose a tag to compare

@zot zot released this 21 Oct 18:03
· 11 commits to main since this release

A minor bump because one signature breaks: FilterByRatio gains a parameter.
Everything else is additive, and the fused AddFile/Reindex/AppendChunks
family behaves as before.

Breaking:

  • FilterByRatio(maxRatio) becomes FilterByRatio(maxRatio, minCount)
    (9310fba, R675). A trigram is skipped only when it is both
    non-discriminating (over maxRatio) and expensive to scan (at least minCount
    postings). minCount of 0 or 1 reproduces the old behaviour exactly.

New:

  • Two-phase indexing (3bb59d5, R676-R689). PrepareFile/PrepareContent/
    PrepareAppend compute chunks without a transaction, from any goroutine;
    StorePrepared/ReindexPrepared/AppendPrepared store them. The opaque
    single-use handles PreparedFile and PreparedAppend join the two. The store
    allocates fileids, derives the reindex removal set in its own transaction,
    and rejects an append whose tail moved (ErrAppendTailMoved).
  • Per-chunk compute (06492ec, R690-R699). SplitFile/SplitContent/SplitAppend
    return SplitChunk{Chunk, Hash} plus FileMeta; PrepareChunk computes one
    chunk's grams, content-pure and cacheable by hash; AssembleFile/
    AssembleAppend package PreparedEntry values into the handles the store
    accepts. Positions always come from the entry's chunk, so reused grams are
    safe across files and scans.
  • SplitAppend takes AppendOption and honours WithBaseLine (8362b4a, R700), so
    appended chunks get absolute line ranges. Existing four-argument callers are
    unchanged.

Also: mini-spec config moves to .minispec/config.toml.