Skip to content

Version 4.0.0

Latest

Choose a tag to compare

@StormBytePP StormBytePP released this 11 Nov 10:20

[Summary]

StormByte Buffer is the byte-buffer module of the StormByte C++ suite.

It depends on StormByte Base 3.0.0, StormByte-Logger 3.0.0 and StormByte-System 3.0.0. Base is its own archive: Logger and System no longer bring it. This repository is not Base, Config, Crypto, Database, Logger, Multimedia, Network or System.

Public headers under StormByte/buffer/ cover FIFO, SharedFIFO, Ring, Producer/Consumer, Hopper, Sink, Bridge, Pipeline and StormByte::Buffer::IO (buffered binary sources and sinks). Octet payloads are StormByte::Safe::Binary. Byte lengths are StormByte::ByteSize. Hopper and Sink count items with StormByte::Size.

If you landed here from a release link and have not read the tree:

  • What this module is, how to build it, and short examples: README.md
  • License: GNU Lesser General Public License version 3 or later, LICENSE

Added

  • Optional allocation-free AdmissionWaitObserver on weighted Hopper/Sink
    PushFor calls reports paired transitions around actual admission waits,
    distinguishing local capacity from shared budget pressure without changing
    admission policy or accounting. Callbacks run on the caller thread outside
    Buffer locks; local bounds take precedence over shared-budget pressure.
  • ByteBudget::SharesIdentity compares shared budget state without
    reserving, locking or changing counters, including after Close(); moved-from
    handles share no identity. Tests cover copies, independent, closed,
    zero-limit, unlimited, moved and concurrent comparison.
  • Configurable ReadChunk, universal WriteTransfer, optional page
    WriteThreshold, atomic live IO configuration and non-I/O effective-transfer
    snapshots. Instrumented tests verify real origin requests, aggregation,
    bounded page payload, partial transfers and concurrent reconfiguration.
  • ReadChunk is independent of prefetch: omitted or zero selects 64 KiB;
    public reads and peeks can assemble multiple bounded origin transfers.
  • WriteTransfer caps every origin push, including direct writes, ring drains,
    page eviction, Flush and Close; omitted or zero is unlimited. WriteChunk
    retains its separate ring-unit/trigger role.
  • WriteThreshold optionally drains contiguous bounded pages in the caller's
    thread to half the effective high watermark. Memory pressure, Flush, Close
    and reconfiguration override the watermark and flush residual data.
  • Generic QueueLimits, weighted TryPush / bounded PushFor, QueueStatus,
    Snapshot, CancelPending and hopper Abort, usable through Hopper<T>
    and existing Sink<T> buckets. Optional item and byte limits apply
    independently; engaged zero is an active zero limit, absence is unlimited.
  • Shared ByteBudget with immediate/bounded reservation, close/cancellation
    and RAII Lease charges for queues and concurrent work. Explicit lease
    copies share one aggregate charge across payload references; individual
    queues still count their own logical entry weights.
  • Queue and budget snapshots distinguish current logical bytes/reservations
    from limits, peaks and backpressure statistics. Queries perform no I/O or
    capacity waits; these weights are not physical-memory or RSS measurements.
  • Tests for shared budgets across buckets, co-writers and work; independent
    limits, live reduction/wakeup, oversize rejection even under local pressure,
    cancellation, EOF/Abort, destruction and
    allocation-failure rollback. Installed and extracted-ZIP consumers exercise
    the exported APIs through a separate shared-library provider.

Removed

  • Breaking: removed the protected scalar-policy constructors of
    BufferedReader, BufferedWriter, BufferedLocationReader and
    BufferedLocationWriter, and the old scalar constructor symbols of the file
    leaves. Derived coordinators pass Parameters, a complete configuration or
    resolved location knobs instead.
  • Breaking: removed virtual dispatch for reader ReadAhead, MaxMemory
    and MaxWait, and writer WriteChunk, BackPressure, MaxMemory and
    MaxWait. Their getters/setters remain available as nonvirtual operations;
    configuration always goes through the coordinator's coherent policy.
  • Breaking: removed binary compatibility with previous IO parameter-bag,
    location-knob and Hopper/Sink template layouts.

Changed

  • IO live setters serialize with session operations, apply a complete policy,
    and may block. Writer changes flush using the old policy; failure keeps that
    policy installed but cannot roll back origin I/O. Void setters throw
    Buffer::Exception on failure; Configure returns an explicit result.
  • New IO policy/snapshot fields follow PascalCase. Complete policies transport
    wait limits as scalar MaxWaitMilliseconds; new budget DLL calls likewise
    transport scalar durations, keeping clock types and deadlines provider-local.
  • Page mode no longer allocates a dormant ring. Ring snapshots distinguish
    logical admission capacity from actual power-of-two storage including a
    spare slot; transactional replacement temporarily accounts for both rings.
  • Preserve Capacity(key, 0) as unlimited and legacy Push as an unweighted,
    nonthrowing operation. New weighted admissions preserve the item on failure
    and report oversize weights without enqueuing or waiting indefinitely.
  • Queue EOF preserves entries and charges until Pop/destruction; Abort discards
    them. A shared lease stays charged until its last copy is released, including
    copies intentionally retained by a caller or concurrent job. Bounded waits
    and wake-all notifications do not promise strict FIFO producer fairness.

Fixed

  • Resize writer rings on reconfiguration, propagate flush failures, preserve
    nonseekable Peek/retry bytes, check queue arithmetic, and avoid whole-page
    block-writing copies. See README for memory accounting and policy semantics.
  • Prevent ring storage growth while its worker borrows a drain span; retain
    partial-transfer progress on errors and avoid duplicate origin-byte counts.
  • Release queue payloads/charges outside queue locks, reserve before weighted
    insertion, and roll back both storage and charge on recoverable allocation
    failure without consuming the caller's item.