Skip to content

Releases

ringfire 0.5.3 — 98% line coverage gate, strict Linux IPC, and clean CLI

Choose a tag to compare

@alex09x alex09x released this 28 Sep 03:21

ringfire 0.5.3 raises the production test coverage gate to 98%, strictly enforces Linux IPC architecture across the codebase, and introduces the clean CLI subcommand.

• Production Rust line coverage above 98%: 98.04% measured on Linux AArch64 (5,214 / 5,318 production lines covered), enforced by CI and scripts/coverage.sh --fail-under-lines 98.
• Strict Linux-only architecture: Top-level compile_error! on non-Linux platforms prevents unsupported usage; removed incomplete macOS/BSD stubs and fallback paths from futex and membarrier primitives.
• New CLI subcommand clean: ringfire clean [PATH] [--dry-run] audits and cleans orphaned shared memory rings and blackboards with no active flock holders or live readers.
• Extended behavioral test suites: Added comprehensive test coverage for MPMC ticket contention/recovery, SPMC sparse skip-hole transitions with reader registries, and FFI error states.
• API additions: Exposed RingProducer::cleanup_mode and RingProducer::set_cleanup_mode for dynamic lifecycle configuration.
• Package sync: Updated Python package bindings to version 0.5.3.

Wire formats and shared memory layout remain backwards-compatible with v0.5.x.

ringfire 0.5.2 — verified benchmarks and 95% coverage gate

Choose a tag to compare

@alex09x alex09x released this 27 Sep 03:27
0f738ca

ringfire 0.5.2 corrects benchmark methodology and makes test coverage a CI requirement.

  • Production Rust line coverage above 95%: 96.67% measured on Linux AArch64 and 96.47% on GitHub Linux x86-64, including CLI, FFI, Tokio and replication. Tests are excluded from the denominator; production modules are included. CI runs on Linux x86-64 and AArch64 and publishes JSON/HTML reports.
  • Receive benchmarks now time successful validated receives, with refill outside the timer and correct batch accounting. IPC compares full 64-byte replies uniformly and separates adaptive futex from zero-spin waiting. Every harness has bounded execution and isolated temporary files.
  • Network examples report distinct-message losses, duplicates and ordering, keep warm-up counters separate, and fail on transport/clock-probe errors. Local and round-trip stage measurements use a monotonic clock.
  • The multiprocess stress test now proves that eight child readers each consume and validate all 100,000 records. Added behavioral CLI/FFI, malformed-input, asynchronous and replication recovery tests.
  • New finite CLI modes: top --iterations N and TCP serve --once. Existing long-running behavior and signal semantics remain unchanged.
  • README charts and tables use three repeated measurements of all 28 Criterion cases; raw samples and machine settings are included. Historical invalid receive figures are withdrawn. Historical LAN/WAN results remain explicitly labelled and were not remeasured.

Benchmark methodology · All measurements and raw samples · Coverage contract

The 95% gate is line coverage, not exhaustive branch or concurrency coverage. The ARM measurements ran on a shared host; every repeat and its range is retained. Wire formats remain compatible with v0.5.x.

Closes #6 via #7.

ringfire 0.5.1 — illustrated network mirrors

Choose a tag to compare

@alex09x alex09x released this 27 Sep 00:43
9bce14e

The network mirror guide now ships with seven SVG diagrams and a dependency-free generator: message pipeline, LAN multicast, WAN site hubs, protocol exchange, stage latency, WAN percentiles, and frame pacing.

The figures were checked against the measured results. This release corrects GAP/NAK directions, distinguishes host topology from the 16-process test, shows per-mirror median ranges, removes an unsupported pacing point, and labels the actual pacing settings. SVG generation explicitly uses UTF-8.

Library behavior, APIs, and wire protocols are unchanged from v0.5.0.

Validated with the existing Linux x86-64 and AArch64 CI suites, package verification, figure rendering, and generator reproducibility.

ringfire 0.5.0

Choose a tag to compare

@alex09x alex09x released this 26 Sep 19:41
a9f7abc

Network mirrors: a ring on one host, identical copies with the same sequence numbers on others.

  • ringfire serve / ringfire mirror and the replication module (ReplicaServer, Mirror, MulticastConfig): records travel as raw slot bytes, so one pair works for any element type; BlobProducer rings (variable-length payloads) are mirrored too.
  • Three transports: TCP; UDP multicast (one datagram for every mirror on a LAN); UDP unicast with NAT punching for other sites and clouds, with --dup N to send every datagram N times. Lost datagrams are recovered from the source ring over TCP (NAK/GAP); a mirror is written strictly in sequence order, never reordered or duplicated.
  • A mirror can be served again, so one copy per site crosses the WAN and a hub redistributes locally.
  • FLAG_SPARSE (0x0400): readers on mirror rings skip holes instead of waiting on them.
  • Frame pacing (--linger-us, adaptive by default) keeps the datagram rate near 20k/s under load.
  • Measured: 30 µs push-to-read across a 1 GbE LAN to each of six mirrors; 16 mirrors at p99 72 µs over multicast; 0 lost / 0 reordered from 1k to 1M msg/s; Tokyo → Los Angeles UDP p99 50 µs above its median where TCP pays a full extra round trip for 1 record in 100. Details, diagrams and recipes: docs/replication.md.

Full changelog: CHANGELOG.md.

v0.4.0 - Correctness Hardening & Wire Protocol v2

Choose a tag to compare

@alex09x alex09x released this 24 Sep 20:33
7d633ae

Performance (AMD Ryzen 9 7950X, 64-byte messages)

64-byte round trip, two threads Time
ringfire, busy-spin 0.32 µs
ringfire, FutexWait 2.27 µs
Unix domain socket 4.76 µs
Pipe 4.93 µs
TCP loopback 10.28 µs

try_recv 12.4 → 6.4 ns, recv_batch(32) 276 → 74 ns, push 1.76 → 1.88 ns (v0.3.0 → v0.4.0, same machine).

Wire protocol v2: v0.4.0 peers refuse to attach to v0.3.x rings (and vice versa) with VersionMismatch. Upgrade producers and consumers together.

Correctness release. The v0.3.0 review found torn reads, hangs, file truncation and
layout bugs across the Rust core, FFI, C header, Python bindings and CLI; each fix below
has a regression test in tests/regression_tests.rs.

Breaking

  • Wire protocol v2 (RINGFIRE_VERSION = 2): v1 and v2 readers/writers refuse each other.
    • Writers mark a slot SLOT_WRITING (u64::MAX) before overwriting its payload.
    • RingHeader gains slots_offset (taken from padding; header is still 128 bytes).
      Rust, FFI, C and Python readers locate slots through it instead of guessing.
  • LayoutSignature hashes the type name without module paths, so the same struct defined
    in two crates matches. Signatures differ from v0.3.0 (irrelevant across the protocol bump).
  • MpmcQueueConsumer delivery is documented and implemented as at-most-once: overrun
    items are dropped and counted (dropped_count()) instead of hanging the consumer.
  • ReaderRegistry::reader_lag / headroom (and the RingProducer/BlobProducer
    wrappers) take the last published sequence and count unread messages consistently;
    BlobProducer::reader_lag was off by one.
  • Attaching a RingConsumer to a LosslessBackpressure ring whose registry is full now
    fails with NoAvailableReaderSlots instead of silently leaving the reader unprotected.
  • ringfire_blackboard_read (FFI) returns -2 and BlackboardConsumer::read returns
    RingfireError::WriterStalled when a slot stays mid-write for over 100 ms.

Fixed

  • Torn reads (all readers): a reader one slot short of being lapped accepted payloads
    the writer was overwriting (1875 torn out of 16.9M reads in 3 s on a 7950X). Fixed by
    the v2 slot marker plus acquire/release fences for AArch64.
  • BlobConsumer hang: recv/view spun forever once the arena wrapped before the
    descriptor ring did. Lapped payloads are now skipped and counted; descriptors are
    bounds-checked before the arena is touched.
  • Live ring truncation: a second create on a live path truncated the file before
    failing the flock, so the running producer and readers hit SIGBUS. All producers
    (ring, blob, MPMC, blackboard, FFI, Python) now lock first; a stale ring from a dead
    producer is replaced by a fresh inode so orphaned readers stay consistent.
  • push_batch ignored LosslessBackpressure.
  • Lossless producer cost: every push scanned the registry and checked each
    reader's liveness via /proc (about 830 ns per push+recv). The producer now caches the slowest
    cursor and rescans only when approaching it or every half ring; liveness checks run only
    while blocked. Cost is on par with lossy mode.
  • MPMC: queue consumers hung on overwritten tickets; concurrent producers one lap apart
    could interleave payloads in a slot. Slots are now taken over by CAS in lap order; a slot
    that stays mid-write for over 10 ms is never taken over (its writer may just be
    descheduled): the waiting producer drops its message instead of tearing the payload.
  • Reader registry: reclaiming a dead reader could wipe a registration that had just
    replaced it (non-CAS store); update_cursor used Relaxed, letting the producer
    overwrite a slot on AArch64 before the reader finished copying it.
  • Blackboard: writer/reader fences for AArch64; readers no longer spin forever on a
    slot left odd by a crashed writer.
  • Attach validation: truncated or foreign files are rejected (CorruptLayout) instead
    of being read out of bounds; the header magic is published last.
  • Python: BlobConsumer read the arena mask from the reserved counter (wrong
    payloads for most offsets) and never checked for arena lapping; added
    try_recv_copy. Producers take an exclusive flock.
  • C header / FFI consumer ignored the reader registry offset and read registry bytes
    as slots on lossless and blob rings. The C header struct is updated to the v2 layout.
  • CLI: dump printed the header as slots on rings without a registry; the retained
    window was off by one; JSON output did not escape strings; prune used non-CAS stores.
  • AsyncRingConsumer as Stream spawned a Tokio task per pending poll and busy-looped
    while idle; it now yields a few times and then waits on a timer.
  • FutexWait lost wake-ups: a consumer going to sleep could miss a message published at
    the same instant and sleep for the full timeout (50 ms by default; about 1 in 1000 round
    trips in a ping-pong, averaging 53 µs per round trip). Fixed with an asymmetric barrier:
    producers register with membarrier(REGISTER_GLOBAL_EXPEDITED) and a consumer issues
    membarrier(GLOBAL_EXPEDITED) before sleeping, so the producer hot path stays barrier-free.
    Futex ping-pong round trip: 2.3 µs (Unix socket: 4.8 µs). Sleeps stay bounded as a fallback
    (timeout: None = 10 ms) for producers that cannot register.
  • SPMC producers wake all sleeping readers (broadcast), not just one.
  • CycleStamp on AArch64 reads cntvct_el0 (it was a +1 counter); added
    CycleStamp::counter_frequency_hz().
  • ConsumerStartMode::Sequence(0) no longer reports a phantom lapped message.

Added

  • tests/regression_tests.rs (19 tests), examples/quickstart.rs (README snippets, run in CI).
  • benches/ipc_compare.rs: 64-byte round trip over ringfire (spin and futex), Unix socket,
    pipe and TCP loopback, measured with one harness; results at the top of the README.
  • rust-version = "1.88" (edition 2024, let-chains).
  • GitHub Actions CI: Linux x86-64 and macOS AArch64, clippy with and without tokio.

v0.2.0 - Consumer Start Modes & Lock-Free SHM Offset Checkpointing

Choose a tag to compare

@alex09x alex09x released this 21 Sep 23:34

Features

  • ConsumerStartMode: Latest (HFT zero backlog), Head (future only), Oldest (replay full buffer), Sequence(seq).
  • Lock-Free SHM Offset Checkpoints: Cache-aligned 64-byte atomic slot in /dev/shm (<10ns commit, zero syscalls, zero disk I/O).
  • RingConsumerBuilder: Fluent builder pattern with auto-type inference.
  • Runtime seeking & committing: seek(seq), commit_offset(), last_processed_sequence().
  • Python ctypes bindings: Full support for start_mode and offset checkpointing in python/ringfire.

v0.1.0 - Zero-Copy Lock-Free IPC Ring Buffer & Shared Memory Bus

Choose a tag to compare

@alex09x alex09x released this 21 Sep 06:05

Changelog

All notable changes to ringfire will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

[0.1.0] - 2026-09-21

Added

  • Core SPMC Ring Buffer: Lock-free single-producer multi-consumer ring buffer in /dev/shm with 128-byte cache-line aligned RingHeader, atomic Acquire/Release synchronization, and zero heap allocations.
  • MPMC Ring Buffer: Multi-producer support with atomic ticket reservation (claim_seq) and competing worker-queue consumers (MpmcQueueConsumer).
  • LatestWins Overflow Policy: Non-blocking writer with zero stalls; lagged readers detect lapping (RecvStatus::Lapped), track dropped metrics, and jump cleanly to active surviving stream frames.
  • Tear-Read Memory Safety: Two-phase seqlock validation ensuring zero torn reads during concurrent writer wraps.
  • Wait Strategies:
    • BusySpin: sub-20ns memory spin loop.
    • YieldBackoff: adaptive OS thread yielding.
    • Futex: Linux kernel sleep/wake with 0% CPU idle usage and atomic waiting_consumers guard (0 syscalls during active streaming).
  • Tokio Async Integration (AsyncRingConsumer):
    • recv().await and recv_batch(&mut [T]).await.
    • Adaptive spinning $\to$ cooperative tokio::task::yield_now().await $\to$ async sleep, eliminating worker thread starvation.
    • Implementation of futures_core::Stream.
  • Shared State Blackboard:
    • $O(1)$ state snapshot table (BlackboardProducer, BlackboardConsumer) in /dev/shm.
    • Per-slot 64-bit seqlock with 64-byte cache line alignment.
    • Benchmarked at 4.09 ns read and 1.19 ns write on AMD Ryzen 9 7950X.
  • Polyglot Bindings:
    • Pure C11 header include/ringfire.h with standalone inline functions and C-ABI export.
    • Zero-dependency Python package python/ringfire using mmap and ctypes.Structure.
  • Hardware Benchmarks: Comprehensive Criterion suite verified on AMD Ryzen 9 7950X (booster), achieving 588M msg/s push throughput and 83M msg/s recv throughput.