Skip to content

Commit

Permalink
import to new repo
Browse files Browse the repository at this point in the history
  • Loading branch information
sabledb committed Mar 30, 2024
0 parents commit 566c857
Show file tree
Hide file tree
Showing 70 changed files with 14,037 additions and 0 deletions.
7 changes: 7 additions & 0 deletions .gitignore
@@ -0,0 +1,7 @@
.codelite/
.ctagsd/
Cargo.lock
rocksdb.db/
target/
tests/
sabledb.db/
165 changes: 165 additions & 0 deletions BENCHMARK.md
@@ -0,0 +1,165 @@
# Benchmarks

---


- Server machine: `AWS`'s `c6gd.4xlarge` ( `16 vCPU + 32 GB of memory` )
- Client machine: `AWS`'s `c6gn.4xlarge` ( `16 vCPU + 32 GB of memory` )
- Both clients are placed in the same `AWS` region and `AZ`
- Benchmarking tool `redis-benchmark` compiled from sources (in release mode)


## `set` command. Payload size: `64`, `128` and `256`

```bash
./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t set -d 64

throughput summary: 448792.75 requests per second
latency summary (msec):
avg min p50 p95 p99 max
1.125 0.328 1.127 1.279 1.423 9.511


./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t set -d 128

Summary:
throughput summary: 420433.06 requests per second
latency summary (msec):
avg min p50 p95 p99 max
1.207 0.312 1.199 1.375 1.551 10.191

./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t set -d 256

Summary:
throughput summary: 363543.81 requests per second
latency summary (msec):
avg min p50 p95 p99 max
1.389 0.728 1.375 1.567 1.799 9.759
```

## `get` command. Payload size: `64`, `128` and `256`

- Stop the server
- Clear the database folder
- Fill the database with keys with the requested size

```bash
## database values are all of size 64
./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t get

Summary:
throughput summary: 950751.12 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.508 0.192 0.495 0.671 1.087 2.647

## database values are all of size 128
./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t get

Summary:
throughput summary: 907441.00 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.524 0.192 0.511 0.695 1.111 2.967

## database values are all of size 256
./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t get

Summary:
throughput summary: 907605.75 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.532 0.192 0.519 0.711 1.119 2.847

```

## `mset` command. Payload size `64`, `128` and `256`

- Each `mset` command is equivalent of `10` `set` commands

```bash
./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t mset -d 64

Summary:
throughput summary: 116535.18 requests per second
latency summary (msec):
avg min p50 p95 p99 max
4.373 1.136 3.871 4.623 6.367 99.519

./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t mset -d 128

Summary:
throughput summary: 71874.19 requests per second
latency summary (msec):
avg min p50 p95 p99 max
7.108 1.616 6.375 7.039 54.111 94.271

./redis-benchmark --threads 16 -c 512 -n 10000000 -r 100000 -h 172.31.13.191 -t mset -d 128

Summary:
throughput summary: 41552.40 requests per second
latency summary (msec):
avg min p50 p95 p99 max
12.226 0.432 7.279 7.935 226.943 267.263

```

## `incr` command

The increment command is unique because it uses a "read-modify-update" in order to ensure the atomicity of the action
which in a multithreaded environment causes a challenge

```bash
./redis-benchmark --threads 16 -c 512 -n 1000000 -r 100000 -h 172.31.13.191 -t incr

Summary:
throughput summary: 443655.72 requests per second
latency summary (msec):
avg min p50 p95 p99 max
1.126 0.544 1.127 1.295 1.383 6.071
```

### `ping_inline` command:

---

```
Summary:
throughput summary: 1050640.88 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.450 0.072 0.407 0.775 1.143 3.455
```

With pipeline set to `20`:

```
Summary:
throughput summary: 6653360.00 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.880 0.136 0.855 1.551 1.815 3.135
```

### `ping_mbulk` command:

---

```
Summary:
throughput summary: 1050640.88 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.434 0.072 0.399 0.727 1.127 3.335
```

With pipeline set to `20`:

```
Summary:
throughput summary: 7968127.50 requests per second
latency summary (msec):
avg min p50 p95 p99 max
0.848 0.144 0.807 1.471 1.711 2.887
```

6 changes: 6 additions & 0 deletions Cargo.toml
@@ -0,0 +1,6 @@
[workspace]
members = ["libsabledb", "sabledb", "sb"]
resolver = "2"

[profile.release]
lto = true
164 changes: 164 additions & 0 deletions README.md
@@ -0,0 +1,164 @@
# SableDb

A modern design to persistent `Redis`, written in `Rust`

## Building

### Linux /macOS

```bash
git clone https://github.com/eranif/sabledb.git
cd sabledb
cargo build --release
cargo test --release
```

### Windows

```bash
git clone https://github.com/eranif/sabledb.git
cd sabledb
CFLAGS="-D_ISOC11_SOURCE" cargo build --release
cargo test --release
```

## Running `SableDb`

```bash
./target/release/sabledb
```

Usage:

```bash
$target/release/sabledb [sabledb.ini]
```

## Supported features

- Persistent data using RocksDb - use `SableDb` as a persistent storage using `Redis`'s API
- TLS connections
- Replication using
- Highly configurable, but comes with sensible defaults

## Supported commands

---

### String commands

| Command | Supported | Fully supported? | Comment |
|---|---|---|---|
| append | ✓ | ✓ |
| decr | ✓ | ✓ |
| decrby | ✓ | ✓ |
| get | ✓ | ✓ |
| getdel | ✓ | ✓ |
| getex | ✓ | ✓ |
| getrange | ✓ | ✓ |
| getset | ✓ | ✓ |
| incr | ✓ | ✓ |
| incrby | ✓ | ✓ |
| incrbyfloat | ✓ | ✓ |
| lcs | ✓ | 🗙 | Does not support: `IDX`, `MINMATCHLEN` and `WITHMATCHLEN` |
| mget | ✓ | ✓ |
| mset | ✓ | ✓ |
| msetnx | ✓ | ✓ |
| psetex | ✓ | ✓ |
| set | ✓ | ✓ |
| setex | ✓ | ✓ |
| setnx | ✓ | ✓ |
| setrange | ✓ | ✓ |
| strlen | ✓ | ✓ |
| substr | ✓ | ✓ |


### List commands

| Command | Supported | Fully supported? | Comment |
|---|---|---|---|
| blmove | ✓ |✓ | |
| blmpop | ✓ |✓| |
| blpop | ✓ | ✓ | |
| brpop | ✓ | ✓ | |
| brpoplpush | ✓ | ✓ | |
| lindex | ✓ | ✓ | |
| linsert | ✓ | ✓ | |
| llen | ✓ | ✓| |
| lmove | ✓ | ✓ | |
| lmpop | ✓ | ✓ | |
| lpop | ✓ | ✓ | |
| lpos | ✓ | ✓ | |
| lpush | ✓ | ✓ | |
| lpushx | ✓ | ✓ | |
| lrange | ✓ | ✓ | |
| lrem | ✓ | ✓ | |
| lset | ✓ | ✓ | |
| ltrim | ✓ | ✓ | |
| rpop | ✓ | ✓ | |
| rpoplpush | ✓ | ✓ | |
| rpush | ✓ | ✓ | |
| rpushx | ✓ | ✓ | |

### Generic commands
| Command | Supported | Fully supported? | Comment |
|---|---|---|---|
| del | ✓ |✓ | |
| ttl | ✓ |✓ | |

## Benchmarks

### Command `set`

| Payload size (bytes) | rps | p50 (ms) | p90 (ms) | p99 (ms) |
|---|---|---|---|---|
| 64 | 448K | 1.127 | 1.279 | 1.423 |
| 128 | 420K | 1.199 | 1.375 | 1.551 |
| 256 | 363K | 1.375 | 1.567 | 1.799 |


### Command `get`

| Payload size (bytes) | rps | p50 (ms) | p90 (ms) | p99 (ms) |
|---|---|---|---|---|
| 64 | 950K | 0.495 | 0.671 | 1.087 |
| 128 | 907K | 0.511 | 0.695 | 1.111 |
| 256 | 905K | 0.519 | 0.711 | 1.119 |


### Command `mset`

Note: Each `mset` command is equivalent of `10` `set` commands


| Payload size (bytes) | rps | p50 (ms) | p90 (ms) | p99 (ms) |
|---|---|---|---|---|
| 64 | 116K | 3.8 | 4.6 | 6.3 |
| 128 | 72K | 6.375 | 7.039 | 54.111 |
| 256 | 41.5K | 0.432 | 7.279 | 226.943 |


### Command `incr`

The increment command is unique because it uses a "read-modify-update" in order to ensure the atomicity of the action which in a multithreaded environment causes a challenge

| Payload size (bytes) | rps | p50 (ms) | p90 (ms) | p99 (ms) |
|---|---|---|---|---|
| N/A | 443K | 1.127 | 1.295 | 1.383 |

### Network only (`ping` command)


| Command | rps | pipeline | p50 (ms) | p90 (ms) | p99 (ms) |
|---|---|---|---|---|---|
| ping_inline | 1.05M | 1 | 0.407 | 0.775 | 1.143 |
| ping_inline | 6.65M | 20 | 0.855 | 1.551 | 1.815 |
| ping_mbulk | 1.05M | 1 | 0.399 | 0.727 | 1.127 |
| ping_mbulk | 7.96M | 20 | 0.807 | 1.471 | 1.711 |

---

Command executions can be seen [here][2]

[1]: https://github.com/redis/redis
[2]: https://github.com/eranif/sabledb/blob/main/BENCHMARK.md
41 changes: 41 additions & 0 deletions libsabledb/Cargo.toml
@@ -0,0 +1,41 @@
[package]
name = "libsabledb"
version = "0.1.0"
edition = "2021"

[dependencies]
bytes = "1"
clap = { version = "4", features = ["derive"] }
thiserror = "1"
tokio = { version = "1", features = ["full"] }
tracing = { version = "0", features = ["log"] }
tracing-subscriber = "0"
async-trait = "0"
test-case = "3"
rand = "0"
lazy_static = "1"
serde_json = "1"
rocksdb = "0"
serial_test = "2.0.0"
ctrlc = "3.4.0"
dashmap = "5.5.3"
num-traits = "0.2.17"
bitflags = "2"
crc16 = "0.4.0"
crossbeam-skiplist = "0.1.2"
crossbeam = "0.8.3"
hdrhistogram = "7.5.4"
sled = "0.34.7"
rustls-pemfile = "2.0.0"
tokio-rustls = "0"
pki-types = { package = "rustls-pki-types", version = "1" }
rustls = { version = "0.23", default-features = false, features = ["std"] }
rust-ini = "0.20.0"
serde = { version = "1", features = ["derive"] }

[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
affinity = "0"

[features]
default = ["rocks_db"]
rocks_db = []
7 changes: 7 additions & 0 deletions libsabledb/build.rs
@@ -0,0 +1,7 @@
#[cfg(target_os = "windows")]
fn main() {
println!("cargo:rustc-link-lib=pthread");
}

#[cfg(not(target_os = "windows"))]
fn main() {}

0 comments on commit 566c857

Please sign in to comment.