Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ exclude = [
"examples/sled-cluster",
"examples/three-nodes-cluster",
"examples/single-node-expansion",
"examples/quick-start",
"benches/d-engine-bench",
]

Expand All @@ -26,7 +27,7 @@ d-engine-core = { path = "./d-engine-core" }
d-engine-client = { path = "./d-engine-client" }
d-engine-server = { path = "./d-engine-server" }

tokio = { version = "1.4", features = [
tokio = { version = "1", features = [
"macros",
"time",
"signal",
Expand Down
6 changes: 3 additions & 3 deletions MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This guide covers **two major breaking changes** in v0.2.0:

### What Changed

Starting from **v0.2.0**, the WAL (Write-Ahead Log) format for file-based state machines has changed to support **etcd-compatible TTL semantics**.
Starting from **v0.2.0**, the WAL (Write-Ahead Log) format for file-based state machines has changed to support **absolute expiration time semantics**.

**Old Format (pre-v0.2.0):**

Expand All @@ -29,7 +29,7 @@ Entry fields: ..., expire_at_secs: u64 (8 bytes, absolute expiration time in UNI
### Why This Change?

- **Crash Safety**: Absolute expiration time ensures TTL correctness across restarts
- **etcd Compatibility**: Matches etcd's lease semantics (absolute expiry)
- **Deterministic Semantics**: Matches industry-standard lease semantics (absolute expiry)
- **No TTL Reset**: TTL no longer resets on node restart

### Impact
Expand Down Expand Up @@ -105,7 +105,7 @@ grep "WAL" /var/log/d-engine.log
| TTL Storage | Relative (seconds from now) | Absolute (UNIX timestamp) |
| After Restart | TTL resets 🔄 | TTL preserved ✅ |
| WAL Replay | All entries loaded | Expired entries skipped ✅ |
| etcd Compatible | ❌ No | ✅ Yes |
| Expiration Semantics | Relative TTL | Absolute timestamp ✅ |
| Crash Safe | ❌ No | ✅ Yes |

---
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,16 @@
[![CI](https://github.com/deventlab/d-engine/actions/workflows/ci.yml/badge.svg)](https://github.com/deventlab/d-engine/actions/workflows/ci.yml)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/deventlab/d-engine)

**d-engine** is a lightweight and strongly consistent Raft consensus engine written in Rust. It is a base to build reliable and scalable distributed systems. **Designed for resource efficiency**, d-engine employs a single-threaded event-driven architecture that maximizes single CPU core performance while minimizing resource overhead. It plans to provide a production-ready implementation of the Raft consensus algorithm, with support for pluggable storage backends, observability, and runtime flexibility.
**d-engine** is a lightweight distributed coordination engine written in Rust, designed for embedding into applications that need strong consistency—the consensus layer for building reliable distributed systems. Start with a single node and scale to a cluster when you need high availability. **Designed for resource efficiency**, d-engine employs a single-threaded event-driven architecture that minimizes resource overhead while maintaining high performance. It provides a production-ready implementation of the Raft consensus algorithm,
with pluggable storage backends, built-in observability, and tokio runtime support.

---

## Features

- **Single-Node Start**: Begin with one node, expand to 3-node cluster when needed (zero downtime).
- **Strong Consistency**: Full implementation of the Raft protocol for distributed consensus.
- **Tunable Persistence**: DiskFirst for maximum durability or MemFirst for lower latency.
- **Flexible Read Consistency**: Three-tier read model (Linearizable/Lease-Based/Eventual) balancing consistency and performance.
- **Pluggable Storage**: Supports custom storage backends (e.g., RocksDB, Sled, Raw File).
- **Observability**: Built-in metrics, structured logging, and distributed tracing.
Expand Down Expand Up @@ -88,7 +91,7 @@ d-engine provides flexible storage abstraction layers. Implement your own storag
- **Custom Storage Engines**: See [Implementing Custom Storage Engines](https://docs.rs/d-engine/latest/d_engine/docs/server_guide/index.html#implementing-custom-storage-engines)
- **Custom State Machines**: See [Implementing Custom State Machines](https://docs.rs/d-engine/latest/d_engine/docs/server_guide/index.html#implementing-custom-state-machines)

Note: For production use, a minimum of 3 nodes is required to ensure fault tolerance.
Note: Single-node deployment is supported for development and low-traffic production. For high availability, you can dynamically expand to a 3-node cluster with zero downtime.

---

Expand Down
Loading
Loading