Version 2.0.0

[Summary]

StormByte Buffer is the byte-buffer module of the StormByte C++ suite.

It depends on StormByte Logger and StormByte System, which bring Base. This repository is not Base, Config, Crypto, Database, Logger, Multimedia, Network or System.

Public headers under StormByte/buffer/ cover FIFO, SharedFIFO, Ring, Producer/Consumer, Hopper, Sink, Bridge, Pipeline and StormByte::Buffer::IO (buffered binary sources and sinks). Octet payloads are StormByte::BinaryData. Byte lengths are StormByte::ByteSize. Hopper and Sink count items with StormByte::Size.

If you landed here from a release link and have not read the tree:

• What this module is, how to build it, and short examples: README.md
• License: GNU Lesser General Public License version 3 or later, LICENSE

Added

• option(BUILD_SHARED_LIBS "Build shared libraries" ON) in the project root. Shared is the default so a consumer can redistribute without triggering LGPL static-link obligations. Static is opt-in (-DBUILD_SHARED_LIBS=OFF). CI passes -DBUILD_SHARED_LIBS=ON. Third-party StormByte pins pass ENABLE_TEST=OFF.
• Nested Parameters on all buffered reader/writer levels, with knobs ReadAhead, MaxMemory, MaxWait, WriteChunk and BackPressure. Omitted knobs retain the previous defaults; omitting all device knobs makes Setup() probe. Brace-init and named Parameters are supported. Variadic knobs resolve in the caller under STORMBYTE_FORCE_INLINE; the DLL receives only numbers and a probe flag.
• BufferedReader page cache and logical seek. Consumed bytes remain in RAM up to MaxMemory; garbage collection evicts farthest from Tell. Seek updates Tell immediately, avoids moving the origin on a cache hit, and resumes prefetch with one OriginSeek when a later read leaves the cached range. Tell never lies.
• BufferedWriter dirty-page cache and logical seek. Writes are lazy until MaxMemory, Flush or Close; nearby corrections and far-future islands are supported while memory allows. Eviction prefers the oldest dirty page behind the origin cursor. Seek is logical; materializing a page (eviction, flush or close) moves the origin.
• Layered telemetry in StormByte::Buffer and StormByte::Buffer::IO. Read/write counters hold delivered/accepted bytes; IO telemetry adds cache/origin/seek/wait counters. MeanRate is the caller-visible effective rate (ByteSize/s), including cache hits, not device throughput. Telemetry handles are const StormByte::Safe::Shared<…>, stable for the life of the office; accumulators do not reset on close. Public writer Flush and the flush in Close count toward the rate; internal worker/GC drains do not. Flattening provides operator StormByte::Safe::String out of line and caller-side STORMBYTE_FORCE_INLINE operator std::string().
• BufferedReader::Available(). Contiguous cached bytes at Tell. Does not call OriginPull / OriginSeek and does not wait for prefetch.
• StormByte::Buffer::Pumper. Takes a Bridge by move and runs Passthrough on a worker until EoF or failure. Starts in the constructor; the destructor joins. Nested Parameters with knobs Chunk and HighWater. Chunk 0 is automatic cycle size, not Bridge “current contents”. HighWater applies to the input only: omitted = 0 if the source is IO, otherwise the backend default (constexpr in the PIMPL .cxx); explicit 0 = no Pumper cap (intended when the IO source already limits itself). Non-IO sources are unbounded by design. Toggle pauses/resumes. Cancel is terminal (Failed, no restart). Telemetry is forwarded from the owned Bridge.
• StormByte::Buffer::Pipe (pipe.hxx / pipe.cxx). Abstract stream stage (StormByte::Safe::Clonable + StormByte::Safe::Unique<Pipe>). Copyable and movable (special members out of line). Run(ReadOnly&, WriteOnly&, const Shared<Logger::Log>&). Clone and Move are public and must be implemented by the leaf with Unique::MakePointer in the leaf TU.

Changed

• Breaking: Port Buffer to StormByte Base 2.0.0. Owned text is now StormByte::Safe::String / Safe::WString; pointer and clonable APIs use StormByte::Safe; Buffer telemetry derives from Base StormByte::Telemetry and measures operations with its named clocks; Buffer exceptions accept Base-owned Safe::String messages.
• Breaking: StormByte::Buffer::Data is gone. Octet payloads are StormByte::BinaryData from Base. data.hxx / data.cxx and DataTests are removed.
• Breaking: byte counts are StormByte::ByteSize (FIFO, Ring, SharedFIFO, Producer / Consumer, Bridge, Pipeline, IO). Hopper<T> and Sink<T> count items with StormByte::Size (Capacity, Size, Buckets, Select).
• Breaking: AvailableBytes() is Available(). The return type is already StormByte::ByteSize.
• Breaking: ExternalReader, ExternalWriter, ExternalBufferReader and ExternalBufferWriter are removed. Bridge borrows non-IO ReadOnly / WriteOnly tips directly; those buffers must outlive it. IO leaves remain owned by move.
• Breaking: Pipeline::PipeFunction (std::function over External reader/writer types) is gone. A stage is a user leaf of Pipe. Pipeline::Add(const Pipe&) clones onto Base's heap and does not touch the caller; Pipeline::Add(Pipe&&) takes Move(). There is no boxing of callables and no public Unique<Pipe> add. Pipeline is stream buffers only (ReadOnly / WriteOnly); IO joins through Bridge / Pumper. Process(Consumer, Shared<Logger::Log>, ExecutionMode) — mode last.
• Breaking: BufferedLocationReader and BufferedLocationWriter sit between the engines and the file leaves. A location is file-like: named by Location() (StormByte::Safe::String, owned by Base), always seekable and sized. Path-only Setup() lives here.
  • Device() and the pure OriginDevice() return StormByte::Safe::Shared<StormByte::System::Device> instead of System::Device by value. A leaf may now hand out a System::Device subclass (for example a NIC device whose accessor is not a filesystem path) and the dynamic type survives, so overridden Throughput() / Window() are honoured and the caller can keep the object alive. Build the owner with StormByte::Safe::Shared<StormByte::System::Device>::MakePointer<Leaf>(…) so the object lives on Base's heap and crosses the DLL boundary safely.
  • New protected virtual bool OriginDeviceUsable(const StormByte::Safe::Shared<StormByte::System::Device>&) const noexcept. The default is the previous behaviour (non-empty owner and operator bool() true, which probes the stored path). A leaf whose identifier is not a filesystem path overrides it and never reaches the non-virtual path probe. An empty owner is always unusable.
  • Setup() stays final, calls OriginDevice() once, asks OriginDeviceUsable() and only then applies Window() on the dynamic object. When the device is not usable the per-leaf defaults (ReadAhead, or WriteChunk / BackPressure / MaxMemory) are kept. The device is never copied or sliced to the base type.
• Breaking: BufferedFileReader and BufferedFileWriter are final. CreateDevice() is gone. Path() (const String&) and Location() (IO::Location) are set on BufferedReader / BufferedWriter and do not change. A file leaf passes Location::Local. A socket on the lower layer can pass Location::Remote. The file leaves keep the plain System::Device and the real path probe, so their windows and defaults are unchanged.
• Breaking: IO constructors no longer take positional windows (read_ahead, max_memory, write_chunk, back_pressure, max_wait). One constructor per leaf: path plus that class’s Parameters (default {} = probe). Explicit zeros stay zeros; they do not probe.
• Breaking: Buffer::Exception uses Exception::Path{"Buffer"}. what() is StormByte.Buffer: message; ReadError and WriteError use StormByte.Buffer.Read and StormByte.Buffer.Write. Buffer-specific destructors are defined in this module.
• Breaking: Bridge is a manual transfer again, not a worker. Public Passthrough(ByteSize, Operation) is the only transfer; Operation::{Blocking, NonBlocking} applies to the read tip; write TryAgain is retried until that call completes. n == 0 is current contents (Available()). Non-IO tips are ReadOnly& / WriteOnly&. IO tips are stolen by move as the concrete leaf. Failed() is sticky. Continuous pumping is Pumper.
• Reader Seek is no longer “always OriginSeek”. A cache hit is O(1) on the origin. A miss still costs a real seek plus whatever the device does.
• Writer Seek exists and is part of the public contract. It is not guaranteed O(1) when the target is not in the dirty map or when eviction must drain pages first.
• Writer contract: lazy write up to MaxMemory. More random access needs more MaxMemory or islands get evicted (a real write + seek).
• Nested BufferedReader::Telemetry / BufferedWriter::Telemetry structs are gone. Counters live on the Shared objects; getters, not public fields.
• LockFreeRing::FrontSpan returns a snapshot copied under the wait mutex so a concurrent Grow cannot invalidate the pointer the drain worker is pushing.
• Origin I/O on the writer (OriginSeek / OriginPush / OriginFlush / OriginOpen / OriginClose / OriginTruncate) is serialized against the drain worker. Flush waits until the ring is empty and the worker has published the origin cursor (!m_drain_run).
• Dual license layout: LICENSE is the short header text; COPYING.LGPLv3 is the LGPL text.

Fixed

• Writer drain vs Grow: FrontSpan no longer aliases m_storage while the producer reallocates (Mac patev-ring-only corruption).
• Writer Flush returning before m_origin_pos was stored, which let the next EnsureOrigin land a patch on the wrong offset.
• Concurrent FILE* / ofstream use from the writer thread and the drain worker.
• Origin cursor after OriginFlush treated as untrusted until the next EnsureOrigin (Darwin). Sequential drain after that first realign does not seek again.
• LockFreeRing Close, SetError, Clean, Drop and Consume publish under the wait mutex. A parallel pipeline stage waiting on an intermediate ring could miss the wake and leave Process spinning on IsWritable().
• Doxygen: broken \ref on the public reader header; private storage types not listed as public API.
• Missing virtual destructors on BufferedLocationReader / BufferedLocationWriter. Leaves stay final and do not declare virtual on the destructor.

Tests

• BufferedFileReaderTests / BufferedFileWriterTests / BridgeTests construct IO with Parameters / knobs (ReadAhead, MaxMemory, WriteChunk, BackPressure). Path-only still probes.
• Predictable hex fixture, integrity of every Read after logical and cold seeks, Tell during a logical seek, telemetry prints via *Telemetry().
• Writer close/flush integrity on hex files, holes, far islands, eviction + patch, ring-only / pages / direct knobs.