Skip to content

v0.9.0

Choose a tag to compare

View all tags
@clemlesne clemlesne released this 14 Jun 22:03
v0.9.0
e581bee

Per-file indexing cap 10 MiB → 100 MiB. New byte-weighted read semaphore caps reader-buffer RAM at ~600 MiB. Drop-in upgrade from v0.8.0 — no CLI or cache-layout changes.

Highlights

  • maxIndexedDocumentBytes 10 MiB → 100 MiB. Vendored libraries, minified bundles, generated JSON/CSV, lockfiles, large protobufs now indexable. Overflow files skipped with slog.Warn.
  • Bounded in-flight memory. A new byte-weighted semaphore caps total bytes resident in reader buffers across all workers at ~600 MiB, so the larger per-file cap does not balloon RSS under bursts of large files.
  • Auto-scaling ceiling. maxInFlightBytes = inFlightHeadroomFiles (6) × maxIndexedDocumentBytes. Bumping the per-file cap scales the in-flight budget for free. Tune inFlightHeadroomFiles for tighter or looser ceilings.

Development

  • New cmd/seek/read_semaphore.go plus lifecycle tests covering leak-free drain on builder errors, panic safety, and ctx cancellation mid-read.
  • Reader weight is released by the consumer after builder.Finish() returns, because Zoekt's shard writers retain Content references until then.
  • New dep: golang.org/x/sync v0.20.0.

Notes

  • Synchronous folder-delta reads stay unbounded by the semaphore (documented as future work).

Full changelog: v0.8.0...v0.9.0

Assets 12
Loading