Skip to content

v0.29.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Apr 01:39
v0.29.0
ff7d397

search-in: full FTS5 operator support, per-block evaluation

Before 0.29.0, search-in walked the manifest with substring matching: only "exact phrase" and bare term matches were honored, FTS5 operators (NEAR/<n>, AND, OR, NOT, prefix*) were treated as literal tokens, and 2-letter substrings (e.g. ho) flooded the result list by hitting prefixes of unrelated words like households.

search-in now indexes one FTS5 row per block in addition to the existing per-document index. Queries run through SQLite's FTS5 parser end-to-end and are evaluated per block — a returned passage individually satisfies the query, not just an approximation derived from a document-level match.

What changed for users

  • The same keyword syntax search accepts now works in search-in too: "exact phrase", OR, NOT, term NEAR/<n> term, prefix*. Boolean operators must be uppercase (FTS5 rule — lowercase or is a literal term).
  • search-in '"Author, Year. Title"' correctly surfaces a citation that wraps across paragraph breaks in the references list, while still respecting token boundaries (no more partial-token over-matches like "lpha beta" hitting inside alpha beta...).
  • Reference-only blocks are no longer deprioritized in search-in. "Does this book cite X?" queries now return the bibliography entry at the top.
  • search-in '"Acemoglu, Daron. 2012"' and the bare search-in 'Acemoglu' both behave correctly: the first matches the exact bibliography form, the second catches both in-text "(Author Year)" parentheticals and the bibliography entry.

Sync impact

The keyword index schema bumped to v3-blockindex. On the next zotagent sync, the keyword index is rebuilt from the existing cached manifests — no PDF re-extraction, no re-embedding (qmd vectors are unaffected). If you skip sync, the first search-in call after upgrade lazily rebuilds the per-block index in the background.

Skill update

The zotagent skill ships an updated FTS5 operator table, a "Drill into one paper with search-in" workflow section, and notes on the contract differences between search (one passage per matched item) and search-in (per-block matches). Run npx skills add TomBener/zotagent to update.

Full Changelog: v0.28.0...v0.29.0

Assets 4
Loading