Skip to content

Version 1.4.0

Choose a tag to compare

@StormBytePP StormBytePP released this 23 Sep 00:15
· 66 commits to master since this release

[Summary]

StormByte Buffer is the byte-buffer module of the StormByte C++ suite.

It depends on StormByte Base and optionally StormByte Logger. This repository is not Base, Config, Crypto, Database, Logger, Multimedia, Network or System.

Public headers under StormByte/buffer/ cover FIFO, SharedFIFO, Ring, Producer/Consumer, Hopper, Sink, Bridge, Pipeline and StormByte::Buffer::IO (buffered binary sources and sinks).

If you landed here from a release link and have not read the tree:

  • What this module is, how to build it, and short examples: README.md
  • License: GNU Lesser General Public License version 3 or later, LICENSE

Added

  • StormByte::Buffer::IO. Buffered binary sources and sinks, separate from FIFO / Ring / Hopper. Public surface: Status, State, Result, ToString, BufferedReader, BufferedWriter, BufferedFileReader, BufferedFileWriter. IO::Backend is the PIMPL and is not a public include.
  • Status / Result / State. Ok, End, Error, Failed, TryAgain plus a byte count. TryAgain is backpressure or a bounded MaxWait. State: Idle, Missing, Directory, Permission, NotWritable, Fault, Unavailable. constexpr ToString for Status and State.
  • BufferedReader. Public base for a binary origin. Leaves implement OriginOpen, OriginClose, OriginPull, OriginCanSeek, OriginSeek, OriginHasSize and OriginSize. Optional Setup() runs once from Open before OriginOpen. Construction is Unavailable; a successful Open is Idle. Close is idempotent; Open is not. operator bool is Idle and not EoF.
  • Reader Read / Peek into a FIFO or a writable std::span<std::byte>. Destination overwritten only on Ok / End with a non-zero count. Empty span is {Ok, 0}. FIFO n == 0 serves the cached span at Tell. MaxWait 0ms waits without limit.
  • Reader Seek / Tell / IsSeekable / IsSized / Size. Absolute or relative only. End-relative is Seek(*Size() + off, Absolute) when sized. Seekable Seek always calls OriginSeek, including a cache hit. Non-seekable Seek is Failed and does not call the hook. Seek is not O(1). Cache is a map of owned spans; overlap merges; MaxMemory evicts farthest from Tell; 0 stores nothing and still serves from the origin. Prefetch stops on Seek and on move (Rebind); the next Read / Peek requests it again.
  • BufferedFileReader. ifstream leaf, seekable and sized. Path-only constructor probes device throughput at Setup and sets ReadAhead (window clamped 16 KiB–1 MiB) with MaxMemory 1 MiB. Explicit (path, read_ahead, max_memory) keeps those knobs. Does not open in the constructor.
  • BufferedWriter. Public base for a binary sink. Leaves implement OriginOpen, OriginClose, OriginPush, OriginFlush and OriginTruncate. Optional Setup() and WillWrite. No Seek. Tell is bytes accepted since Open or Truncate. Close flushes then closes; a flush failure is Fault.
  • Writer Write(const FIFO&), Write(FIFO&) and Write(std::span<const std::byte>). Atomic. WriteChunk and BackPressure (in chunks): either knob 0 is direct; both > 0 use an SPSC LockFreeRing capped at BackPressure * WriteChunk bytes. Overflow is TryAgain. Dirty() is unread ring bytes. Flush() drains the ring and calls OriginFlush.
  • BufferedFileWriter. ofstream leaf, binary append. Creates the file when the parent exists (no mkdir -p). Path-only constructor probes the device at Setup and sets WriteChunk plus BackPressure 4. Explicit (path, write_chunk, backpressure) keeps those knobs. Truncate overwrites.
  • Device throughput probe (private): Linux / Windows / macOS classification (HDD, SATA SSD, NVMe gen, USB, network at 80 % of NIC). Nominal rates, not a benchmark. Device knobs have no setters; MaxMemory and MaxWait stay settable.
  • LockFreeRing::FrontSpan, Consume and Write(std::span<const std::byte>).
  • ExternalWriter::Occupied.
  • Bridge pumps any ExternalReader / IO reader into any ExternalWriter / IO writer. Drain respects sink backpressure. Worker auto-drains; public Passthrough is gone. high_water == 0 means no extra occupancy cap. The worker starts, including when high_water is 0. Pause is only Drainer(Toggle).
  • Two-argument Bridge constructors for BufferedWriter sinks:
    Bridge(const IO::BufferedReader&, IO::BufferedWriter&) and
    Bridge(ExternalReader&, IO::BufferedWriter&). No occupancy cap at
    the Bridge layer. Same pump path as high_water == 0. Intended for
    BufferedFileWriter (WriteChunk / BackPressure already cap Dirty).
    Pairings into FIFO / SharedFIFO / Ring / Producer keep the
    three-argument constructor.

Removed

  • Sink::Bind and Sink::Bind(int, Sink&). Wire with To(key) / >> / <<.

Tests

  • BufferedFileReaderTests. Fixtures under test/files/. Span Read / Peek, Tell, Seek (absolute, relative, end via Size, cache hit, MaxMemory 0), path-only vs explicit constructors, move with prefetch stopped.
  • BufferedFileWriterTests. Temp files via StormByte::System::TempFileName. Direct (path, 0, 0), path-only device knobs, Dirty / Flush / BackPressure / Truncate / move.
  • BufferedMeteredFileTests. Selective override example (BytesRead / BytesWritten).
  • Bridge coverage for pipe close-while-started, high_water 0, and the two-argument writer ctors (test_io_uncapped_ctor, test_buf_to_io_uncapped_ctor).