Skip to content

v0.7.0

Choose a tag to compare

@bluesky585 bluesky585 released this 17 Sep 15:51
· 16 commits to main since this release
09ec9b9

Retrieval becomes a complete story: chunk, embed, persist on either front end, rank three ways.

Store

  • SQLite index store (#58): nibble -index docs.db writes chunks plus vectors to a SQLite database — one row per chunk, keyed by text and offsets so a re-upserted chunk updates in place. Vectors are float32, halving the file with the ranking preserved. Incremental: an upsert writes the rows it touched, not the whole file, and the file reopens across processes. Same exact brute-force cosine ranking as the JSONL store; the driver is pure Go, no CGO.

Chunking

  • Recursive overlap (#59): recursive accepts -overlap — the next chunk repeats the previous chunk's tail in its own text, the way token windows widen, so a retrieval hit on either side of a cut can see across it. Chunks stay exact slices of the source; the last still ends where the input does.

HTTP API

  • Scoring modes (#60): POST /v1/search takes scoring (dense default, bm25, hybrid) and hybrid_weight, matching the CLI. Only dense and hybrid embed the query; bm25 needs no embedder at all. The ranking lives in pkg/store, shared by both front ends, so they cannot drift. The TypeScript client carries the same options.
  • Persistent index (#61): nibble-api -index docs.db backs the API's index with the same SQLite file shape the CLI writes — created when missing, resumed by the next server on the same path. Without the flag, the memory default is unchanged.

Full change list

  • #58 store: SQLite index store with incremental upsert
  • #59 recursive: overlap repeats the previous chunk's tail
  • #60 httpapi: scoring modes for /v1/search, shared with the CLI
  • #61 httpapi: -index backs the API's index with a SQLite file
  • #62 Pin the install commands to v0.7.0