Skip to content

v0.1.8

Choose a tag to compare

View all tags
@chonknick chonknick released this 23 Mar 05:03
· 24 commits to main since this release
v0.1.8
9ed3db6

What's Changed

Reranking Performance Overhaul (3.3x faster)

  • Single rerank pass — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads)
  • ORT format — pre-optimized flatbuffer model (reranker.ort) for faster deserialization, with automatic fallback to .onnx
  • mmap model loading — zero-copy memory-mapped file access via commit_from_memory_directly
  • Multi-threaded inference — uses half available cores (capped at 4) for ONNX intra-op parallelism

Benchmarks

Scenario v0.1.7 v0.1.8 Speedup
All packages (11) 1.01s 0.31s 3.3x
Single package 0.09s 0.04s 2.3x
Assets 9
Loading