v0.3.1
v0.3.1 — Chunk change deltas on re-ingest (BL-010)
First Phase 2 bridge item: the MuninnDB bridge (and any WatchDocuments consumer) now gets incremental chunk-delta updates on document re-ingest instead of full re-processing.
New in v0.3.1
RE_INGESTED event (spec 043 / BL-010):
- A document re-ingest (via
Reprocess,ReprocessAll, or the watcher's MODIFIED path) now emits a singleRE_INGESTEDevent on theWatchDocumentsstream, carrying a per-chunk delta:ADDED/REMOVED/UNCHANGED. - The delta is computed by content identity (a new per-chunk
ContentHashsidecar = SHA-256 of the chunk's text), so a moved paragraph isUNCHANGEDand repeated text collapses correctly (multiset diff). RE_INGESTEDreplaces theINGESTED(new)+DELETED(old)pair a re-ingest previously surfaced (no double-counting).- The event carries an old→new chunk-ID map (
prev_chunk_idonUNCHANGED/REMOVEDdeltas), so consumers can remap stored references (no orphaned bookmarks).
Embed-skip (US2):
UNCHANGEDchunks whose embedding model hasn't changed skip the expensive Ollama embedding call — the oldPrefixEmbeddingis copied to the new chunk ID. A model change forces re-embedding (no stale vectors).- ACK budget verified: the diff computation is 13.7µs for a 50-chunk doc (730× under the <10ms budget).
v2 schema migration:
- Backfills
ContentHashon existing chunks (ExpectedVersion1→2). No new storage prefix; the field rides in the existingPrefixChunkJSON value.
Bridge docs:
- Upstream
#560(idempotent_id wiring) shipped by the MuninnDB maintainer;#556(UPSERT) now unblocked at the prerequisite level. - Phase 2 scoping notes recorded (BL-010/011/013 assessed against the codebase).
Security / quality
- 5-agent adversarial review of the implementation (concurrency, correctness, embed-skip safety, identity stability, event contract): no fatal flaws; all HIGH + MEDIUM findings fixed.
go test -race ./...full repo green;make lint0 issues.
Full Changelog: v0.3.0...v0.3.1