Repository navigation
v0.9.0
This release completes the retrieval story around the chunker: index management, filtered search, and a faster, half-sized vector pipeline.
Source management (#67)
Indexed chunks can now carry a source — a file path, a URL, a caller-chosen key.
- CLI:
-sourcewith-index,-list-sources,-delete-source - API:
sourceon the index body,GET /v1/sources,DELETE /v1/sources/{name} - Chunks are keyed by text and offsets, so re-indexing a document under a new source moves it there — replace without rebuilding the index. Deleting to zero is reported as 0, not an error.
Source-filtered search (#68)
- CLI:
-sourceon-query; API: optionalsourceon the search body. - The filter lands before scoring, so
kcounts filtered hits. Absent searches everything; an empty string matches the records indexed without a source.
float32 vectors end to end (#69) — breaking change for Go consumers
embed.Embedder.Embed now returns [][]float32, and Chunk.Embedding / Record.Vector are []float32. Scoring runs over the stored bytes directly: memory halves and the per-query float64 conversion is gone.
- The SQLite file format is unchanged — existing databases open as they are. The JSONL and wire formats are unchanged; the TypeScript client needs no update.
- The cosine scan gained 4-lane accumulators (the profiler showed accumulator serialization, not memory bandwidth, was the bound): 200k chunks at 1536 dimensions went 591ms → 297ms; 50k went 143ms → 81ms.
- Ranking agreement with the previous float64 scoring was checked over 500 randomized trials — no top-5 disagreement.
rag-search example (#70)
go run ./examples/rag-search walks the retrieval lifecycle in one program: chunk with a real chunker, index into SQLite under per-document sources, hybrid search, source-filtered search, delete a source, and search again.
Full changes: v0.8.0...v0.9.0