Skip to content

1.5.0

Choose a tag to compare

@DavidBelicza DavidBelicza released this 27 Jul 17:07
faf746e
  • Chunks are now buffered across documents and sent to the embedding server in batches instead of one request per document, cutting HTTP round trips on remote embedding endpoints.
  • Added IndexOptions.EmbedBatchSize to control the batch size. Leave it nil for the default of 50; set a positive value to override (1 sends one chunk per request). Zero or negative is rejected.
  • Batches contain whole documents and succeed or fail as a unit. A failed batch leaves its documents in the chunked state for the next run to retry.
  • Added IndexOptions.OnProgress to report indexing progress as (phase, done, total) across the scanning, indexing, and cleanup phases.
  • Added the examples/progress program showing progress reporting during an index run.
  • Documented batching, change detection, and file identity behavior in docs/architecture.md.
  • Raised the fingerprint and process document page sizes from 1 to 100, reducing per document database round trips during indexing.
  • Expanded test coverage of storage, search, and fingerprint error paths.