Skip to content
πŸ”‘A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.
Go Makefile Other
Branch: master
Clone or download
jsign Test for data corruption in datafile decoding (#99)
* internal/data: move codec to own subpackage

* internal/data/codec: check & test nil Entry Decode

* internal/data/decoder: test for short prefix error

* internal/data/codec: test invalid key & value sizes

* internal/data/codec: check & test for truncated data

* interna/data/codec: use assert for tests
Latest commit f17187a Sep 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Create FUNDING.yml Aug 25, 2019
cmd Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019
internal Test for data corruption in datafile decoding (#99) Sep 16, 2019
tools Fixed typo in release.sh (#8) Apr 25, 2019
.drone.yml Increased read performance by ~3-4x by removing another unnecessary I… Mar 23, 2019
.gitignore Add a simple Redis compatible server daemon (bitcaskd) Mar 13, 2019
.goreleaser.yml Fixed injecting Version/Commit in relased binaries Mar 19, 2019
AUTHORS Added AUTHORS file to record contributors beyond the scope of Github … Aug 7, 2019
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md Mar 12, 2019
CONTRIBUTING.md Update CONTRIBUTING.md Aug 5, 2019
Dockerfile Add Dockerfile to publish images to Docker Hub Mar 30, 2019
LICENSE Create LICENSE Mar 12, 2019
Makefile Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019
README.md Update to Go 1.13 and update README with new benchmarks (#89) Sep 4, 2019
_config.yml Set theme jekyll-theme-hacker Mar 12, 2019
bitcask.go Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019
bitcask_test.go Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019
doc.go custom high-performance encoder implementation (#52) Aug 7, 2019
doc_test.go custom high-performance encoder implementation (#52) Aug 7, 2019
go.mod Update to Go 1.13 and update README with new benchmarks (#89) Sep 4, 2019
go.sum Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019
options.go Makefile setup & key/value coherent datatypes & refactoring (#98) Sep 12, 2019

README.md

bitcask

Build Status CodeCov Go Report Card codebeat badge GoDoc GitHub license Sourcegraph

A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak

For a more feature-complete Redis-compatible server, distributed key/value store have a look at Bitraft which uses this library as its backend. Use Bitcask as a starting point or if you want to embed in your application, use Bitraft if you need a complete server/client solution with high availability with a Redis-compatible API.

Features

  • Embeddable (import "github.com/prologic/bitcask")
  • Builtin CLI (bitcask)
  • Builtin Redis-compatible server (bitcaskd)
  • Predictable read/write performance
  • Low latency
  • High throughput (See: Performance )

Development

$ git clone https://github.com/prologic/bitcask.git
$ make

Install

$ go get github.com/prologic/bitcask

Usage (library)

Install the package into your project:

$ go get github.com/prologic/bitcask
package main

import "github.com/prologic/bitcask"

func main() {
    db, _ := bitcask.Open("/tmp/db")
    defer db.Close()
    db.Put("Hello", []byte("World"))
    val, _ := db.Get("Hello")
}

See the godoc for further documentation and other examples.

Usage (tool)

$ bitcask -p /tmp/db set Hello World
$ bitcask -p /tmp/db get Hello
World

Usage (server)

There is also a builtin very simple Redis-compatible server called bitcaskd:

$ ./bitcaskd ./tmp
INFO[0000] starting bitcaskd v0.0.7@146f777              bind=":6379" path=./tmp

Example session:

$ telnet localhost 6379
Trying ::1...
Connected to localhost.
Escape character is '^]'.
SET foo bar
+OK
GET foo
$3
bar
DEL foo
:1
GET foo
$-1
PING
+PONG
QUIT
+OK
Connection closed by foreign host.

Docker

You can also use the Bitcask Docker Image:

$ docker pull prologic/bitcask
$ docker run -d -p 6379:6379 prologic/bitcask

Performance

Benchmarks run on a 11" Macbook with a 1.4Ghz Intel Core i7:

$ make bench
...
goos: darwin
goarch: amd64
pkg: github.com/prologic/bitcask

BenchmarkGet/128B-4         	  316515	      3263 ns/op	  39.22 MB/s	     160 B/op	       1 allocs/op
BenchmarkGet/256B-4         	  382551	      3204 ns/op	  79.90 MB/s	     288 B/op	       1 allocs/op
BenchmarkGet/512B-4         	  357216	      3835 ns/op	 133.51 MB/s	     576 B/op	       1 allocs/op
BenchmarkGet/1K-4           	  274958	      4429 ns/op	 231.20 MB/s	    1152 B/op	       1 allocs/op
BenchmarkGet/2K-4           	  227764	      5013 ns/op	 408.55 MB/s	    2304 B/op	       1 allocs/op
BenchmarkGet/4K-4           	  187557	      5534 ns/op	 740.15 MB/s	    4864 B/op	       1 allocs/op
BenchmarkGet/8K-4           	  153546	      7652 ns/op	1070.56 MB/s	    9472 B/op	       1 allocs/op
BenchmarkGet/16K-4          	  115549	     10272 ns/op	1594.95 MB/s	   18432 B/op	       1 allocs/op
BenchmarkGet/32K-4          	   69592	     16405 ns/op	1997.39 MB/s	   40960 B/op	       1 allocs/op

BenchmarkPut/128BNoSync-4   	  123519	     11094 ns/op	  11.54 MB/s	      49 B/op	       2 allocs/op
BenchmarkPut/256BNoSync-4   	   84662	     13398 ns/op	  19.11 MB/s	      50 B/op	       2 allocs/op
BenchmarkPut/1KNoSync-4     	   46345	     24855 ns/op	  41.20 MB/s	      58 B/op	       2 allocs/op
BenchmarkPut/2KNoSync-4     	   28820	     43817 ns/op	  46.74 MB/s	      68 B/op	       2 allocs/op
BenchmarkPut/4KNoSync-4     	   13976	     90059 ns/op	  45.48 MB/s	      89 B/op	       2 allocs/op
BenchmarkPut/8KNoSync-4     	    7852	    155101 ns/op	  52.82 MB/s	     130 B/op	       2 allocs/op
BenchmarkPut/16KNoSync-4    	    4848	    238113 ns/op	  68.81 MB/s	     226 B/op	       2 allocs/op
BenchmarkPut/32KNoSync-4    	    2564	    391483 ns/op	  83.70 MB/s	     377 B/op	       3 allocs/op

BenchmarkPut/128BSync-4     	     260	   4611273 ns/op	   0.03 MB/s	      48 B/op	       2 allocs/op
BenchmarkPut/256BSync-4     	     265	   4665506 ns/op	   0.05 MB/s	      48 B/op	       2 allocs/op
BenchmarkPut/1KSync-4       	     256	   4757334 ns/op	   0.22 MB/s	      48 B/op	       2 allocs/op
BenchmarkPut/2KSync-4       	     255	   4996788 ns/op	   0.41 MB/s	      92 B/op	       2 allocs/op
BenchmarkPut/4KSync-4       	     222	   5136481 ns/op	   0.80 MB/s	      98 B/op	       2 allocs/op
BenchmarkPut/8KSync-4       	     223	   5530824 ns/op	   1.48 MB/s	      99 B/op	       2 allocs/op
BenchmarkPut/16KSync-4      	     213	   5717880 ns/op	   2.87 MB/s	     202 B/op	       2 allocs/op
BenchmarkPut/32KSync-4      	     211	   5835948 ns/op	   5.61 MB/s	     355 B/op	       3 allocs/op

BenchmarkScan-4             	  568696	      2036 ns/op	     392 B/op	      33 allocs/op
PASS

For 128B values:

  • ~300,000 reads/sec
  • ~90,000 writes/sec
  • ~490,000 scans/sec

The full benchmark above shows linear performance as you increase key/value sizes.

Stargazers over time

Stargazers over time

Support

Support the ongoing development of Bitcask!

Sponser

Contributors

Thank you to all those that have contributed to this project, battle-tested it, used it in their own projects or products, fixed bugs, improved performance and even fix tiny typos in documentation! Thank you and keep contributing!

You can find an AUTHORS file where we keep a list of contributors to the project. If you contriibute a PR please consider adding your name there. There is also Github's own Contributors statistics.

License

bitcask is licensed under the term of the MIT License

You can’t perform that action at this time.