Repository navigation
v0.3.0
The built-in full-text index (R2) is rebuilt as incremental segments, so it keeps working on large vaults. No host-side code changes are needed; existing vaults migrate automatically on their first access after deployment.
- The index is now a set of immutable segments. Each indexing pass writes only the notes that changed into one segment (at most 2 million code units per pass, a few seconds), so the cost of an update is proportional to the change, not to the vault. The initial indexing is split into passes the same way and stays under the Durable Object CPU and memory limits (measured: a 100 MB vault of 12,720 notes indexes in about 2 minutes).
- Idle passes merge small segments automatically (compaction, without re-tokenizing).
- Ranking is now BM25 over the query phrases (it used to be matches / log2(length), with no IDF).
- The previous whole-vault generation is retired and deleted automatically once every note is in a new segment.
indexStatus().fts.pending(notes not yet in the built-in index) is added.VaultBindings.ftsMaxTotalCodeUnitsnow bounds the whole index (default 50,000,000 code units ≈ 120 MB of Japanese Markdown; in 0.2.0 it bounded one rebuild at 8,000,000). Exceeding it is still reported inindexStatus().fts.error.- A rebuild that keeps dying mid-way is recorded in
indexStatus().fts.errorinstead of being retried forever (up to 0.2.0 it retried every 60 seconds). - Omitting
VaultBindings.vectorize/embedderruns without semantic search (no embeddings, no Vectorize). In this repository's Worker that is theSEMANTIC_SEARCH=offvariable.
R2 layout: fts/{tenant}/{db}/manifest.json plus seg-*/. A manifest.json written by 0.2.0 or earlier (version 1) is still read.