v0.5.1

TinyStore 0.5.1

What's new, with code in every language: https://tinyshed.org/tinystore/docs/releases/v0.5.1

bun add @tinyshed/tinystore@0.5.1
npm install @tinyshed/tinystore@0.5.1
pip install tinyshed-tinystore==0.5.1

Built for linux-x64, linux-arm64, linux-x64-musl, linux-arm64-musl, darwin-arm64, darwin-x64, win32-x64. Each archive holds the
tinystore executable in bin/, and in lib/ the core's library and the Node addon; the Go
SDK fetches its platform's at the first open. SHA256SUMS checks them all.

Changes since v0.5.0

Fixes

• search: keep a key's directory in chunks of 256 blocks, so packing writes its last chunk rather than the whole, which set ms marco's third million passages twice as slowly as its first

Performance

• search: unpack a block a word at a time, read lengths only for postings scored, walk windows forward and complete a document lazily, so ms marco's median search takes 3.1 ms rather than 5.3 and its p99 21 rather than 34
• search: merge ten segments of a level as lucene's tiers do, so ms marco's 8.8 million passages build in 293 s with two sets in flight
• search: look a set's words up once, prepare a merge's statements once a commit and read its keys' parts a few statements a step, so ms marco's 8.8 million passages build in 5 minutes with two sets in flight
• search: pack merges in a snapshot beside their commits, keep the keys' ids sets met, skip composing text already in nfkc, take words between spaces whole, and checkpoint a large search.db less often, so ms marco's 8.8 million passages build in 6 minutes
• search: keep postings in segments a set writes one after another and merges join, so a set writes pages for its postings rather than one for each of its words, and ms marco's 8.8 million passages build in 13 minutes
• search: keep the head a row a key a set, leave replaced documents out of packing, and keep only longer keys' best as varints
• search: keep keys' directories between searches, start each search from its keys' ten best, and give a dropped index's pages back to the disk
• search: find a set's keys by their term and count them by their id, with no returning, a set three times sooner
• search: keep postings bit-packed in runs read through blob handles, walk passive lists' bounds and write a set_many's postings at once

Full history: v0.5.0...v0.5.1.