Repository navigation
v0.5.0 — cloud-sync throughput batch
The reframing release: segment-buffer is a single-process throughput buffer for cloud sync, with configurable durability, optional performant encryption, and at-least-once delivery.
Migration (breaking changes)
Four breaking changes, batched so you upgrade once. All are API-surface changes — the on-disk format is unchanged; existing segment files (including encrypted and legacy monitor365 files) read without migration.
1. SegmentConfig.cipher: Box → Arc
// before (v0.4.x)
config.cipher = Some(Box::new(cipher));
// after (v0.5.0)
use std::sync::Arc;
let config = SegmentConfig::builder().cipher(Arc::new(cipher)).build();Why: makes SegmentConfig and its builder Clone (the cipher Arc is shared between clones).
2. SegmentError::Io: path: Option<PathBuf> → site: IoSite
// before (v0.4.x)
match err {
SegmentError::Io { path, source } => { /* path: Option<PathBuf> */ }
}
// after (v0.5.0)
match err {
SegmentError::Io { site, source } => {
// site: IoSite — Dir | Segment(PathBuf) | Unknown
}
}Why: None was overloaded (directory-level failure vs no context attached). IoSite makes every case explicit. with_path still works; new with_dir tags directory-level failures.
3. SegmentStore::write_atomic signature (loom feature only)
// before (v0.4.x)
fn write_atomic(&self, range: SegmentRange, bytes: Vec<u8>) -> Result<()>;
// after (v0.5.0)
fn write_atomic(&self, range: SegmentRange, bytes: Vec<u8>, policy: DurabilityPolicy) -> Result<()>;Who is affected: only callers who implement SegmentStore externally (the trait is reachable only under the loom Cargo feature). The production RealStore is constructed internally by open().
4. SegmentConfig new field: durability: DurabilityPolicy
#[non_exhaustive] struct — direct struct-literal construction was already forbidden; builder callers are unaffected. Internal struct-literal callers must add durability: DurabilityPolicy::Segment (or Default::default()).
What's new
Configurable durability
use segment_buffer::{DurabilityPolicy, SegmentConfig};
// Cloud-sync: skip fsync (cloud is the durable layer)
let config = SegmentConfig::builder()
.durability(DurabilityPolicy::Throughput)
.build();
// Standalone queue: maximum crash safety
let config = SegmentConfig::builder()
.durability(DurabilityPolicy::Maximal)
.build();| Policy | Fsync file | Fsync dir after rename | Worst-case crash loss |
|---|---|---|---|
Maximal |
yes | yes | last in-flight flush only |
Segment (default) |
yes | no | rename window (~5–30s, kernel-dependent) |
Throughput |
no | no | entire OS dirty window (~30s); cloud is durable |
Throughput is ~12% faster than Segment, ~26% faster than Maximal on bench_durability_policy.
Single-process lock (flock)
open() now acquires an exclusive flock on <dir>/.segment-buffer.lock. If another process holds it, open() returns SegmentError::Locked { path } immediately (no block, no timeout). The lock is released on Drop.
XChaCha20-Poly1305 cipher
use segment_buffer::SegmentConfig;
// recommended_cipher(key) installs XChaCha20Poly1305Cipher
let config = SegmentConfig::builder()
.recommended_cipher(&key)
.build();24-byte extended nonce (no AES-GCM 2³²-message-per-key limit), constant-time in software (no AES-NI dependency). Legacy AES-GCM segments still decrypt through AesGcmCipher.
Owned-item iterator
for (seq, item) in buf.iter_from(start, 1000)? {
// standard Iterator combinators work
}SegmentIter<'_, T> yields (seq, item) pairs; for_each_from stays for the zero-copy lending path.
Performance
- Pooled write-side zstd
Compressor:append/batch_115.09 µs → 7.75 µs (2.07× faster). Net ~2.3× vs v0.1.0 at small batches. - Pooled read-side zstd
Decompressor: symmetric pooling forread_from/for_each_from. - Latency histogram stress test: p50/p90/p99/p99.9 per-append latency in CI.
Loom-proven concurrency
4 new loom tests exhaustively enumerate every two-thread schedule of the delete_acked + append interleaving, proving head_seq <= pending_start (the "honest backlog" invariant) across all schedules.
Three new examples
cloud_sync.rs— runnable at-least-once drain loop (happy + retry).cloud_sync_disk_full.rs— metrics-not-policy backpressure pattern.idempotent_server.rs— consumer-side(producer_id, seq)dedup pattern.
Full changelog
See CHANGELOG.md [0.5.0] for the complete per-item detail, including internal changes, dependency bumps, and process hardening.
Verification gate (all green on the tagged commit b7904d6):
cargo fmt --all -- --checkcargo clippy --all-targets {,--features encryption} -- -D warningscargo test --no-fail-fast --features encryption— 96 unit/property + 38 doctests passedRUSTDOCFLAGS="-D warnings" cargo doc --no-deps {,--features encryption}RUSTFLAGS="--cfg loom" cargo test --features loom --test loom --release— 9 passed- CI + Nix workflows:
success