Ship native full-text search for Harper v5.3

[kylebernhardy]

Outcome

Deliver @harperfast/fulltext, a production-ready Tantivy Node-API wrapper with native and caller-owned rocksdb-js storage modes. Harper v5.3 will consume only the Rocks-backed mode.

Product boundaries

• Tantivy 0.26.1 is the initial pinned engine.
• @harperfast/fulltext/native uses Tantivy MmapDirectory for standalone use and as a reference oracle.
• @harperfast/fulltext/rocks uses a caller-owned rocksdb-js database and does not require Harper.
• Harper exposes schema-native full-text indexing through its existing schema, Table.search(), and REST mechanisms.
• Harper does not ship a fallback to Tantivy filesystem storage.

Success criteria

• BM25, Boolean term matching, phrase, bounded prefix/autocomplete, bounded fuzzy matching, and configurable highlighting are implemented.
• The Rocks adapter passes Tantivy directory conformance, crash recovery, backup, lifecycle, and mixed-workload tests.
• Multiple logical indexes can ingest and search concurrently within bounded process and database budgets.
• The agreed catalog workload demonstrates end-to-end search p99 below 50 ms.
• CI publishes multi-platform packages with provenance and retains comparable benchmark history.
• README, API documentation, examples, support policy, and Apache-2.0 licensing are complete.
• Derived-index delivery protocol (DerivedIndexBackend): shared post-commit delivery, watermark/replay, and blob-content contract for HNSW and full-text indexes harper#2489 supplies the shared derived-index delivery and replay contract.

Release gate

Harper integration is blocked from release until the Rocks-backed adapter passes its correctness and performance gates. Native filesystem storage is not a Harper release fallback.

Comment generated by kAIle (Codex GPT-5)