Skip to content

v0.2.0

Choose a tag to compare

@zot zot released this 12 Apr 20:12
· 40 commits to main since this release

What's New

Search

  • SearchFuzzy — typo-tolerant search via trigram OR-union tally, no bigram index needed
  • SearchMulti — collect candidates once, score per strategy (map[string]ScoreFunc)
  • WithLoose() — OR across terms (AND within each term)
  • SearchRegex with trigram-accelerated boolean evaluation

tmp:// Overlay

  • In-memory documents searchable alongside the LMDB index
  • AddTmpFile, UpdateTmpFile, AppendTmpFile (shell >> semantics), RemoveTmpFile

ChunkCache

  • NewChunkCache() — per-query cache for file content and chunked data
  • Retrieve() for cached chunk content
  • Copy() and InvalidateCaches() for read/write path separation

Chunkers

  • BracketChunker — table-driven, line-oriented depth tracking
  • IndentChunker — indentation-scoped, reuses BracketLang config
  • MarkdownChunkFunc — honors fenced code blocks
  • ChunkCallback API for streaming chunk processing
  • CopyPairs exported; Attrs surfaced in ChunkResult

Performance

  • NewSearchCache with pathToID reverse cache — >2x search speedup
  • Coalesced T/W batch writes on add

Architecture

  • Single LMDB subdatabase with prefix-distinguished records
  • Chunk deduplication via SHA-256 content hash
  • TxnHolder interface — records carry transaction context
  • searchConfig receiver refactor for cleaner search internals
  • RecordCounts, FileIDPaths, UnmarshalFHeader utilities

Removed

  • Bigram index (too slow, too fat — SearchFuzzy handles the use case)