BanyanDB: bluge index analysisWorker pool not released on segment rotation causes goroutine drift

[hanahmily]

Summary

During a 48-hour soak of standalone BanyanDB, goroutine count grew from 556 to 708 (+27%). Root cause is bluge index writers spawning their analysis-worker pools on each segment rotation without releasing them when the writer goes idle.

Reproduction

1. Run standalone BanyanDB with a measure group using SegmentInterval: 1 day.
2. Drive sustained write traffic for ≥48 h (any continuous write workload that crosses a UTC midnight reproduces; in our setup we used SkyWalking OAP traffic plus a synthetic ~1000-row/day fixture).
3. Sample /debug/pprof/goroutine?debug=1 every 30 min.

Observed pattern

Time (UTC)	Goroutines	Δ	Note
t0	556	—	baseline
t0 + ~21 h (first UTC midnight crossed)	556 → 632	+76	first segment rotation event
t0 + ~45 h (second UTC midnight crossed)	632 → 708	+76	second event, identical shape

The two events are spaced exactly 24 h apart and add ~76 goroutines each. Between events, goroutine count is flat to within ±1 — no steady leak, only a daily step.

Stack analysis

Diffing the goroutine profile between start and end:

• +108 in bluge/index.analysisWorker at github.com/blugelabs/bluge/index.OpenWriter.func1 (writer.go:77 → analysisWorker at writer.go:667).
• The remaining ~44 are orchestration goroutines around the new writers (waiters, transmit loops).
• Every other stack signature (e.g. pkg/flow.Transmit, grpc/internal/grpcsync.CallbackSerializer.run) is identical count start vs end.

The 108 = 2 events × ~54 analysisWorkers per writer. bluge sizes this pool from GOMAXPROCS (host had 32 CPUs, default pool size ~54), so per-event growth scales with the worker host's GOMAXPROCS.

Hypothesis

When a tsTable rotates to a new daily segment, BanyanDB opens a new bluge index writer for that segment but does not close writers for older segments that are no longer being written to. Each leftover writer keeps its analysisWorker goroutine pool alive.

Over weeks/months the growth would be linear in segment count, eventually pressuring the Go scheduler and overall memory footprint.

Suggested fix direction

Close bluge index writers for segments outside the current write window. This may already be the intent of segment-lifecycle hooks in banyand/internal/storage/; the leak suggests a missing close path on the bluge writer specifically.

Environment

• Standalone mode, single-node deployment.
• SegmentInterval: 1 day, default flush timeouts.
• Host: 32-core Linux, no swap.
• Sustained write rate ~1 req/s through gRPC.