v0.11.0
The write path becomes append-only - incremental index folds, inline embed at ingest, and delta-only copy - and remote pond_search drops from ~8s to sub-second.
🛠 Breaking Changes
- Append-only write path.
pond syncnow embeds each message inline in its ingest commit, and both sync and copy fold the scalar indexes incrementally (optimize_indices(append)) instead of a fullcreate_index(replace=true)rebuild - so the whole-source-column index rebuild that dominated each sync/copy tail (~520s on the real corpus) is gone.pond copycarries only absent-or-grown sessions and appends a grown session's delta rows through the shared ingest write path (append, not merge-insert), keeping remote copies bandwidth-bound rather than commit-latency-bound.
🚀 Performance
- search: warm remote
pond_searchdrops from ~7.9s to sub-second (best 224ms) on the full S3 corpus (11,788 sessions / 2.14M messages). Two stages did work the query never needed.has_embeddingsanswered "does this store have embeddings?" with anIsNotNull(vector)scan of the entire vector column (6.8-11.7s per query); it now reads the manifest (index presence, ~0ms) and only falls back to aLIMIT 1probe when no index exists. Per-hit part summaries fetched file blobs from S3 and scannedpartsonce per session sequentially; they now skip the blob (the label ridesvariant_datametadata) and run concurrently. The real retrieval - embed + IVF probe + hydrate - is ~0.1s. - search (#75):
from_date/to_datereturned empty on remote stores because themessages_timestamp_zonemapmis-prunes the tz-awaretimestampcolumn (ScalarValue::partial_cmpacross the tz mismatch prunes every zone). The index is dropped and date bounds run as a refine over the candidate set. Stores that already built it: runpond optimize --drop-index messages_timestamp_zonemaponce, or date filters stay empty there.
📚 Documentation
- add remote read-path cold-start plan and drop stale prewarm comment figures
🧹 Chores
- bench batch/commit sweeps, append-only write-path plan, AIMD hands-off rule
- enforce changelog header taxonomy (pre-commit + moon + CI); backfill 0.10.1/0.10.2
Full Changelog: v0.10.2...v0.11.0