v0.5.0

New opt-in APIs, all additive — nothing existing changes behavior.

Chunker options

• sentencechunker.MinRunes(n) / semantic.MinRunes(n): merge sentence pieces shorter than n runes, absorbing abbreviation fragments such as the e. that e.g. this yields. Opt-in and off by default: in CJK a 2-rune sentence is complete, and merging it with its neighbor destroys a real boundary.
• semantic.SimilarityWindow(n): widen the cut test from adjacent sentence pairs to the mean vectors of n sentences on each side. Smooths single-sentence wording jitter that would otherwise split an unchanged topic. Consecutive below-threshold points are one boundary, cut at the deepest.
• recursive.RulesFromJSON: the rule hierarchy from JSON (-rules file on the CLI, a rules field on the HTTP API). attach accepts only prev, because that is how nibble's split always attaches a delimiter.

Text handling

• overlap.Merge: fold neighbor context into chunk text itself, in prefix, suffix, and justified modes. Breaks the reconstruct guarantee by design; Prefix/Suffix remain for reconstruct-safe context.
• tokenizer.CountBatch: measure many pieces in one call.

Throughput

• batch.Chunk: chunk many texts through a worker pool (min(8, NumCPU*3/4)). Results line up with inputs by index; errors report the lowest failing index. The CLI chunks -dir inputs through it.

Tooling

• Benchmark baselines for the core packages against a fixed corpus (go test -run xxx -bench=. -benchmem ./pkg/...), documented as a manual before/after comparison rather than a CI gate.