You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Opt-in DecoderPool process-wide decode budgets with a validated bon
builder, runtime resizing, stable max-min cross-decoder allocation, fair
contended admission, and aggregate queue/thread/activity telemetry. Existing
decoders retain private elastic scheduling unless a pool is attached.
Persistent runtime growth through DecoderHandle::request_workers and
reader forwarding methods. DecoderStats now exposes the explicit request,
desired concurrency before pool contention, and whether the shared pool is
limiting progress.
A concurrent programmatic-reader benchmark driver for shared versus private
scheduling through both ordinary Read and paraseq FASTQ parsing.
Optional cpu-accounting feature reporting lifetime worker and auxiliary
thread CPU through DecoderStats::completed_worker_cpu_time, completed_auxiliary_cpu_time, and cpu_accounting_failures. Each thread
reads its own CPU clock exactly twice, at registration and exit, so no timing
call, atomic, or branch is added to decode task begin/end or the inflate
loops. With the feature disabled the accounting compiles out and the fields
are None. Threads still running are omitted until exit, making this
component accounting rather than an exact decision-window counter.
Optional busy-time-accounting feature exposing a monotonic cumulative
executing-region integral as DecoderStats::accounted_busy_time. This gives
a scheduler an exact busy-time signal that does not depend on its own polling
cadence; reconstructing the same quantity by sampling executing-worker counts
was measured 17-313% off on sparse, bursty, and stored decoder paths.
Changed
Marker/window admission now uses immutable configured decoder and pool
capacity rather than the transient application ceiling. A low early throttle
bounds screen execution but no longer permanently latches a suitable stream
to the sequential path, allowing reliable later growth.
Pool execution accounting covers specialized workers, sequential chunks,
bounded format scans, coordinator bridges/fallbacks, and ordered checksums.
Permits are released before bounded result and final reader handoffs so a
backpressured decoder cannot strand capacity needed by another file.
Shared BGZF and dense-member workers amortize global permit traffic with a
worker-local lease that is retained only across nonblocking handoffs and
immediately available work. Shared positional readers sample an
allowance-sized logical output backlog only when the pool is divided, so
broad per-file headroom does not multiply persistent parser buffering or
penalize a sole fast reader.