v0.2.4 - Async IO Architecture + Jepsen Validated

🎯 Key Features

Async IO Architecture (#295, #313, #329–#346)

• Raft event loop fully non-blocking — WAL writes, state machine apply, and replication
  all run off the hot path
• AppendEntries uses a persistent bidirectional stream per peer; replication is pipelined
  across followers
• Dedicated raft-io and SM apply threads eliminate contention on the event loop

Watch API Improvements (#379, #380, #294)

• prev_kv: watch events optionally include the previous value before the change
• Progress heartbeat: periodic events confirm the stream is alive — enables reliable
  gap detection at reconnect
• Prefix watch: watch_prefix(prefix) streams all events under a path prefix
• Buffer overflow protection: watcher limits prevent unbounded memory growth (#294)

Prefix Scan (#378)

• client.scan_prefix(prefix) — returns all keys under a prefix with a revision anchor
• Designed for reliable watch reconnect: scan → rebuild state → resume watch,
  filtering events by revision

Cluster Membership Streaming (#327, #328)

• EmbeddedEngine::watch_membership() / GrpcClient::watch_membership() — real-time
  membership change stream in both embedded and standalone modes
• Yields current snapshot on connect, then one event per committed ConfChange

Simpler Startup (#303)

• EmbeddedEngine::start(data_dir) and StandaloneEngine::run(data_dir, shutdown_rx)
  — no config file required for common cases

Jepsen Validated (#369)

• 5 workloads + 6-hour soak test under combined kill/partition/pause faults
• See Correctness Guarantees

---

🚀 Performance

Embedded Mode vs v0.2.3 (AWS EC2 c5.2xlarge, 3-node, 5-round average)

Scenario	v0.2.3	v0.2.4	Δ
HC Write	64K ops/s	110K ops/s	+72% ✅
Linearizable Read	181K ops/s	327K ops/s	+81% ✅
Lease Read	379K ops/s	341K ops/s	-10% ⚠️
Eventual Read	395K ops/s	363K ops/s	-8% ⚠️

⚠️ Lease/Eventual Read show a ~8–10% regression from scheduling overhead introduced
by the async IO architecture. Root cause identified — fix targeted for v0.2.5.
Linearizable Read and Write are unaffected.

Embedded Mode vs etcd 3.2.0 (AWS EC2 3-node)

Scenario	d-engine v0.2.4	etcd 3.2.0	Δ
HC Write	110,798 ops/s	44,341 ops/s	+2.5x ✅
Linearizable Read	327,355 ops/s	141,578 ops/s	+2.3x ✅
Eventual Read	363,407 ops/s	185,758 ops/s	+96% ✅

Standalone Mode vs v0.2.3 (AWS EC2 3-node)

Scenario	v0.2.3	v0.2.4	Δ
HC Write	36K ops/s	59K ops/s	+65% ✅
Linearizable Read	51K ops/s	77K ops/s	+53% ✅
Lease Read	62K ops/s	76K ops/s	+22% ✅
Eventual Read	151K ops/s	170K ops/s	+12% ✅

Standalone Linearizable Read (77K ops/s) is below etcd's 141K ops/s. etcd uses a
read-index optimization that avoids a full Raft round-trip per read; d-engine
currently issues a full consensus round per standalone linearizable request.
A read-index optimization is planned.

See benchmark report for full details.

---

🐛 Fixes

• #390: Fixed Raft lease clock (SystemTime → Instant) — lease validity now uses
  monotonic time, preventing false expiry under clock adjustments. With a correct lease
  implementation, linearizable reads are served locally when the leader holds a valid
  lease (327K ops/s embedded, +81% vs v0.2.3)
• #381: Linearizable reads incorrectly bypassed quorum check in multi-voter clusters
  — now correctly requires majority confirmation
• #371: WriteBatchWithIndex used in apply_chunk to prevent CAS stale-read under
  concurrent apply
• #308: Snapshot install success now driven by apply result, not transfer ACK —
  fixes silent install failures
• #315: Snapshot disk I/O isolated from Raft event loop via spawn_blocking —
  prevents event loop stalls under large snapshots
• #253: Snapshot stale-file corruption on retry fixed; configurable chunk timeout added
• #290: Reduced snapshot cooldown and fixed check throttling
• #294: Watch buffer overflow protection

---

⚠️ Breaking Changes

StateMachine::apply_chunk Signature (#388)

// Before (v0.2.3)
fn apply_chunk(&mut self, entries: Vec<Entry>) -> Vec<ApplyResult>;

// After (v0.2.4)
fn apply_chunk(&mut self, entries: &[ApplyEntry]) -> Vec<ApplyResult>;

Update your StateMachine implementation to accept &[ApplyEntry]. ApplyEntry
contains the same data — this change eliminates an unnecessary allocation per apply batch.

See MIGRATION_GUIDE.md.

Wire protocol is compatible with v0.2.3 — rolling upgrades are safe.

---

📦 Installation

Embedded mode:

[dependencies]
d-engine = "0.2.4"

Standalone mode (gRPC):

[dependencies]
d-engine = { version = "0.2.4", features = ["client"], default-features = false }

---

🧪 Testing & Reliability

• Jepsen validated: 5 workloads + 6-hour soak test under combined kill/partition/pause faults
• 1000+ tests, all passing with cargo nextest run --all-features

---

📚 Resources

• Docs: docs.rs/d-engine
• Migration Guide: MIGRATION_GUIDE.md
• Examples: examples/
• Benchmarks: benches/reports/v0.2.4/
• Crates.io: crates.io/crates/d-engine

---

Full Changelog: v0.2.3...v0.2.4