Benchmark

Environment

Mac mini Mid 2011

Processor 2.3 GHz Intel Core i5
Memory 8 GB 1333 MHz DDR3
Software OS X 10.9.4 (13E28)
SSD 128 GB

Configuration

redis: close save and aof
ssdb: close binlog manually below:

void BinlogQueue::add_log(char type, char cmd, const leveldb::Slice &key){
    tran_seq ++;
    //Binlog log(tran_seq, type, cmd, key);
    //batch.Put(encode_seq_key(tran_seq), log.repr());
}

leveldb：

compression       = false
block_size        = 32KB
write_buffer_size = 64MB
cache_size        = 500MB

boltdb:

nosync = true

Use Redis-Benchmark tool

LedisDB with goleveldb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 28868.36 requests per second
GET: 30156.82 requests per second
INCR: 28042.62 requests per second
LPUSH: 25176.23 requests per second
LPOP: 18288.22 requests per second
LPUSH (needed to benchmark LRANGE): 23052.10 requests per second
LRANGE_100 (first 100 elements): 6175.89 requests per second
LRANGE_300 (first 300 elements): 2640.61 requests per second
LRANGE_500 (first 450 elements): 1817.45 requests per second
LRANGE_600 (first 600 elements): 1331.38 requests per second
MSET (10 keys): 14120.31 requests per second

LedisDB with leveldb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 27886.22 requests per second
GET: 21413.28 requests per second
INCR: 25786.49 requests per second
LPUSH: 21691.97 requests per second
LPOP: 20833.33 requests per second
LPUSH (needed to benchmark LRANGE): 24026.91 requests per second
LRANGE_100 (first 100 elements): 7936.51 requests per second
LRANGE_300 (first 300 elements): 2677.95 requests per second
LRANGE_500 (first 450 elements): 2199.35 requests per second
LRANGE_600 (first 600 elements): 1629.99 requests per second
MSET (10 keys): 20080.32 requests per second

LedisDB with rocksdb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 27654.87 requests per second
GET: 29550.83 requests per second
INCR: 28457.60 requests per second
LPUSH: 26497.09 requests per second
LPOP: 25087.81 requests per second
LPUSH (needed to benchmark LRANGE): 27027.03 requests per second
LRANGE_100 (first 100 elements): 7053.18 requests per second
LRANGE_300 (first 300 elements): 2935.48 requests per second
LRANGE_500 (first 450 elements): 2303.40 requests per second
LRANGE_600 (first 600 elements): 1686.74 requests per second
MSET (10 keys): 15634.77 requests per second

LedisDB with hyperleveldb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 29137.53 requests per second
GET: 25278.06 requests per second
INCR: 26567.48 requests per second
LPUSH: 29515.94 requests per second
LPOP: 27639.58 requests per second
LPUSH (needed to benchmark LRANGE): 28835.06 requests per second
LRANGE_100 (first 100 elements): 8227.74 requests per second
LRANGE_300 (first 300 elements): 3317.63 requests per second
LRANGE_500 (first 450 elements): 2070.91 requests per second
LRANGE_600 (first 600 elements): 1620.75 requests per second
MSET (10 keys): 18677.62 requests per second

LedisDB with lmdb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 28818.44 requests per second
GET: 24497.80 requests per second
INCR: 20938.02 requests per second
LPUSH: 22026.43 requests per second
LPOP: 23419.20 requests per second
LPUSH (needed to benchmark LRANGE): 25575.45 requests per second
LRANGE_100 (first 100 elements): 8325.01 requests per second
LRANGE_300 (first 300 elements): 3686.23 requests per second
LRANGE_500 (first 450 elements): 2617.94 requests per second
LRANGE_600 (first 600 elements): 1889.29 requests per second
MSET (10 keys): 20885.55 requests per second

LedisDB with boltdb

$ redis-benchmark -p  6380   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 2609.60 requests per second
GET: 25290.85 requests per second
INCR: 2549.46 requests per second
LPUSH: 1624.11 requests per second
LPOP: 1239.56 requests per second
LPUSH (needed to benchmark LRANGE): 1257.67 requests per second
LRANGE_100 (first 100 elements): 9889.24 requests per second
LRANGE_300 (first 300 elements): 4054.82 requests per second
LRANGE_500 (first 450 elements): 2812.31 requests per second
LRANGE_600 (first 600 elements): 2106.59 requests per second
MSET (10 keys): 1777.15 requests per second

Redis 2.8

$ redis-benchmark -p  6379   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 28785.26 requests per second
GET: 30211.48 requests per second
INCR: 31826.86 requests per second
LPUSH: 32010.24 requests per second
LPOP: 30978.94 requests per second
LPUSH (needed to benchmark LRANGE): 30506.40 requests per second
LRANGE_100 (first 100 elements): 13793.10 requests per second
LRANGE_300 (first 300 elements): 6505.33 requests per second
LRANGE_500 (first 450 elements): 4856.25 requests per second
LRANGE_600 (first 600 elements): 3964.79 requests per second
MSET (10 keys): 26852.85 requests per second

SSDB latest stable

$ redis-benchmark -p 8888   -n 50000 -t set,incr,get,lpush,lpop,lrange,mset -q

SET: 29832.94 requests per second
GET: 28137.31 requests per second
INCR: 28851.70 requests per second
LPUSH: 29103.61 requests per second
LPOP: 27901.79 requests per second
LPUSH (needed to benchmark LRANGE): 29638.41 requests per second
LRANGE_100 (first 100 elements): 979.76 requests per second
LRANGE_300 (first 300 elements): 355.19 requests per second
LRANGE_500 (first 450 elements): 231.87 requests per second
LRANGE_600 (first 600 elements): 179.98 requests per second
MSET (10 keys): 21729.68 requests per second

Use Ledis-Benchmark tool

LedisDB with goleveldb

$ ledis-benchmark -n 3000 -port=6380

set: 24143.09 requests per second
incr: 25924.69 requests per second
get: 28356.55 requests per second
rpush: 24129.88 requests per second
lrange: 18876.66 requests per second
lrange: 8523.78 requests per second
lrange: 4821.31 requests per second
lpop: 22452.35 requests per second
hset: 25732.27 requests per second
hget: 25563.63 requests per second
hincrby: 23457.92 requests per second
hdel: 21250.97 requests per second
zadd: 19222.70 requests per second
zincrby: 21545.64 requests per second
zrange: 666.01 requests per second
zrangebyscore: 628.49 requests per second
zrevrange: 63.09 requests per second
zrevrangebyscore: 22.90 requests per second
zrem: 23324.34 requests per second

LedisDB with leveldb

$ ledis-benchmark -n 3000 -port=6380

set: 25670.67 requests per second
incr: 26972.25 requests per second
get: 20356.61 requests per second
rpush: 26212.65 requests per second
lrange: 16392.43 requests per second
lrange: 8398.24 requests per second
lrange: 5059.10 requests per second
lpop: 24098.06 requests per second
hset: 26435.01 requests per second
hget: 19914.47 requests per second
hincrby: 24545.21 requests per second
hdel: 22800.48 requests per second
zadd: 20124.14 requests per second
zincrby: 20933.10 requests per second
zrange: 3356.76 requests per second
zrangebyscore: 3083.32 requests per second
zrevrange: 121.32 requests per second
zrevrangebyscore: 45.43 requests per second
zrem: 21198.36 requests per second

LedisDB with rocksdb

$ ledis-benchmark -n 3000 -port=6380

set: 25445.18 requests per second
incr: 27024.74 requests per second
get: 27854.57 requests per second
rpush: 25997.50 requests per second
lrange: 18839.88 requests per second
lrange: 8659.91 requests per second
lrange: 5167.70 requests per second
lpop: 23216.47 requests per second
hset: 26075.90 requests per second
hget: 28183.21 requests per second
hincrby: 24291.93 requests per second
hdel: 21984.06 requests per second
zadd: 20445.58 requests per second
zincrby: 22799.16 requests per second
zrange: 5769.69 requests per second
zrangebyscore: 5652.89 requests per second
zrevrange: 133.57 requests per second
zrevrangebyscore: 113.01 requests per second
zrem: 21606.00 requests per second

LedisDB with hyperleveldb

$ ledis-benchmark -n 3000 -port=6380

set: 26897.13 requests per second
incr: 26133.26 requests per second
get: 21651.17 requests per second
rpush: 25595.54 requests per second
lrange: 17009.30 requests per second
lrange: 7037.65 requests per second
lrange: 4762.64 requests per second
lpop: 21583.59 requests per second
hset: 25608.47 requests per second
hget: 23319.35 requests per second
hincrby: 23062.92 requests per second
hdel: 21364.14 requests per second
zadd: 19372.19 requests per second
zincrby: 25227.29 requests per second
zrange: 3040.68 requests per second
zrangebyscore: 2966.46 requests per second
zrevrange: 109.29 requests per second
zrevrangebyscore: 42.23 requests per second
zrem: 20411.20 requests per second

LedisDB with lmdb

$ ledis-benchmark -n 3000 -port=6380

set: 25949.01 requests per second
incr: 26993.77 requests per second
get: 27270.71 requests per second
rpush: 26403.66 requests per second
lrange: 17132.23 requests per second
lrange: 8690.68 requests per second
lrange: 5070.16 requests per second
lpop: 23844.47 requests per second
hset: 26296.16 requests per second
hget: 26484.43 requests per second
hincrby: 24700.65 requests per second
hdel: 24708.48 requests per second
zadd: 23173.40 requests per second
zincrby: 25020.85 requests per second
zrange: 24993.24 requests per second
zrangebyscore: 24063.70 requests per second
zrevrange: 26075.47 requests per second
zrevrangebyscore: 23769.81 requests per second
zrem: 25144.48 requests per second

LedisDB with boltdb

$ ledis-benchmark -n 3000 -port=6380

set: 2627.21 requests per second
incr: 2565.72 requests per second
get: 27659.74 requests per second
rpush: 2067.23 requests per second
lrange: 19280.82 requests per second
lrange: 8702.46 requests per second
lrange: 4980.21 requests per second
lpop: 1621.86 requests per second
hset: 2441.69 requests per second
hget: 27605.53 requests per second
hincrby: 2350.96 requests per second
hdel: 2419.91 requests per second
zadd: 2329.33 requests per second
zincrby: 2367.86 requests per second
zrange: 27318.40 requests per second
zrangebyscore: 25946.14 requests per second
zrevrange: 26707.56 requests per second
zrevrangebyscore: 26017.10 requests per second
zrem: 2393.94 requests per second

Redis 2.8

$ ledis-benchmark -n 3000 -port=6379

set: 29373.47 requests per second
incr: 29878.24 requests per second
get: 29056.13 requests per second
rpush: 28837.36 requests per second
lrange: 27694.51 requests per second
lrange: 19949.59 requests per second
lrange: 14082.76 requests per second
lpop: 29694.82 requests per second
hset: 29456.31 requests per second
hget: 31020.73 requests per second
hincrby: 29660.13 requests per second
hdel: 29504.82 requests per second
zadd: 29840.35 requests per second
zincrby: 27501.18 requests per second
zrange: 31620.11 requests per second
zrangebyscore: 23824.60 requests per second
zrevrange: 30413.00 requests per second
zrevrangebyscore: 28600.65 requests per second
zrem: 32168.58 requests per second

SSDB latest stable

$ ledis-benchmark -n 3000 -port=8888

set: 25641.61 requests per second
incr: 26682.50 requests per second
get: 24433.75 requests per second
rpush: 26884.05 requests per second
lrange: 7007.23 requests per second
lrange: 2009.80 requests per second
lrange: 984.30 requests per second
lpop: 25404.18 requests per second
hset: 26525.97 requests per second
hget: 23762.91 requests per second
hincrby: 26346.00 requests per second
hdel: 25853.36 requests per second
zadd: 27018.46 requests per second
zincrby: 21097.96 requests per second
zrange: 6128.86 requests per second
zrangebyscore: 6189.67 requests per second
zrevrange: 260.11 requests per second
zrevrangebyscore: 239.80 requests per second
zrem: 25401.04 requests per second

Conclusions

LedisDB backed by leveldb and rocksdb is little slower than Redis or SSDB, some reasons may cause it:

Go is fast, but not faster than c/c++.
LedisDB backed by these storage engines uses cgo to call leveldb, a big cost.

Rocksdb is not faster than what we expect, maybe the reason of not appropriate configuration. Even of the developers of RocksDB admitted that they don't fully understand the effect of each configuration change.
Generally, among the storage engines supported by LedisDB, lmdb performs better than others, espeically zrange* operations.
LedisDB backed by HyperLevelDB performs better than by 'leveldb' and 'rocksdb'.

However, LedisDB is still an alternative NoSQL in production for you.

Benchmark

Pages 11

Clone this wiki locally

Benchmark

Environment

Configuration

Use Redis-Benchmark tool

LedisDB with goleveldb

LedisDB with leveldb

LedisDB with rocksdb

LedisDB with hyperleveldb

LedisDB with lmdb

LedisDB with boltdb

Redis 2.8

SSDB latest stable

Use Ledis-Benchmark tool

LedisDB with goleveldb

LedisDB with leveldb

LedisDB with rocksdb

LedisDB with hyperleveldb

LedisDB with lmdb

LedisDB with boltdb

Redis 2.8

SSDB latest stable

Conclusions