Skip to content
A feature complete and high performance multi-group Raft library in Go.
Go C++ C Makefile HTML CMake Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
binding docs: updated binding/README.md Jul 23, 2019
client dragonboat: moved internal/utils to its own repo Jul 16, 2019
config raft: adjusted inmemory size calculation Jul 21, 2019
docs logdb: some tweaks for the pebble support Oct 6, 2019
examples docs: added an example folder with the example link in it. Apr 13, 2019
internal logdb: some tweaks for the pebble support Oct 6, 2019
logger dragonboat: moved internal/utils to its own repo Jul 16, 2019
plugin dragonboat: enabled Go module support Jun 21, 2019
raftio dragonboat: fixed minor issues reported by static checks Jul 25, 2019
raftpb dragonboat: fixed a data race when requesting leadership transfer Aug 3, 2019
scripts etc: removed scripts/knossosCheck.py Jul 24, 2019
statemachine dragonboat: added AUTHORS file, updated copyright notice Jun 16, 2019
tools dragonboat: fixed minor issues reported by static checks Jul 25, 2019
.codecov.yml etc: not to check patch coverage Jul 13, 2019
.gitignore logdb: added a test. Jun 13, 2019
.travis.yml tests: test go 1.13 and 1.12 in travis-ci Sep 9, 2019
AUTHORS dragonboat: updated the AUTHORS file. Jun 17, 2019
CHANGELOG.md docs: update CHANGELOG.md Jul 23, 2019
CONTRIBUTING.md Updated CONTRIBUTING.md to encourage contributors to use the new draft Feb 22, 2019
LICENSE Initial commit of dragonboat. Jan 3, 2019
Makefile Makefile: added targets for cross compilation Oct 6, 2019
NOTICE docs: Updated the NOTICE file. May 24, 2019
README.CHS.md logdb: some tweaks for the pebble support Oct 6, 2019
README.md logdb: some tweaks for the pebble support Oct 6, 2019
_config.yml Set theme jekyll-theme-cayman Jan 23, 2019
benchmark_test.go dragonboat: updated benchmark_test.go Jul 21, 2019
binding.go binding: remove unused import Jun 21, 2019
event.go metrics: added transport metrics Jul 6, 2019
example_test.go dragonboat: enabled Go module support Jun 21, 2019
execengine.go dragonboat: logging cleanup Jul 22, 2019
execengine_test.go dragonboat: enabled Go module support Jun 21, 2019
feedback.go dragonboat: logging cleanup Jul 22, 2019
feedback_test.go dragonboat: enabled Go module support Jun 21, 2019
go.mod logdb: some tweaks for the pebble support Oct 6, 2019
go.sum logdb: some tweaks for the pebble support Oct 6, 2019
monkey.go drummer: removed drummer from the dragonboat repo Jul 16, 2019
monkeynoop.go dragonboat: added AUTHORS file, updated copyright notice Jun 16, 2019
node.go dragonboat: fixed a data race when requesting leadership transfer Aug 3, 2019
node_test.go dragonboat: minor cleanups Jul 19, 2019
nodehost.go dragonboat: fixed a data race when requesting leadership transfer Aug 3, 2019
nodehost_slow_test.go dragonboat: fixed minor issues reported by static checks Jul 25, 2019
nodehost_test.go logdb: some tweaks for the pebble support Oct 6, 2019
queue.go dragonboat: use a goroutine to handle all leader info update events Jul 1, 2019
queue_test.go dragonboat: use a goroutine to handle all leader info update events Jul 1, 2019
quiesce.go dragonboat: logging cleanup Jul 22, 2019
quiesce_test.go dragonboat: enabled Go module support Jun 21, 2019
requests.go Merge branch 'fix-drop-proposal-data-race' Sep 10, 2019
requests_test.go dragonboat: fixed a data race when requesting leadership transfer Aug 3, 2019
snapshotstate.go dragonboat: minor cleanups Jul 19, 2019
snapshotstate_test.go dragonboat: minor cleanups Jul 19, 2019
snapshotter.go dragonboat: logging cleanup Jul 22, 2019
snapshotter_test.go dragonboat: minor cleanups Jul 19, 2019
trace.go dragonboat: enabled Go module support Jun 21, 2019

README.md

dragonboat

Dragonboat - A Multi-Group Raft library in Go / 中文版

license Build status travis-ci Go Report Card codecov Godoc Join the chat at https://gitter.im/lni/dragonboat

News

  • 2019-07-04 Dragonboat v3.1 has been released, please read CHANGELOG before you upgrade.
  • 2019-06-21 Dragonboat v3.0 has been released with on disk state machine and Go module support (CHANGELOG).
  • 2019-02-20 Dragonboat v2.1 has been released.

About

Dragonboat is a high performance multi-group Raft consensus library in Go with C++11 binding support.

Consensus algorithms such as Raft provides fault-tolerance by alllowing a system continue to operate as long as the majority member servers are available. For example, a Raft cluster of 5 servers can make progress even if 2 servers fail. It also appears to clients as a single node with strong data consistency always provided. All running servers can be used to initiate read requests for aggregated read throughput.

Dragonboat handles all technical difficulties associated with Raft to allow users to just focus on their application domains. It is also very easy to use, our step-by-step examples can help new users to master it in half an hour.

Features

  • Easy to use API for building Raft based applications in Go or C++
  • Feature complete and scalable multi-group Raft implementation
  • Disk based and memory based state machine support
  • Fully pipelined and TLS mutual authentication support, ready for high latency open environment
  • Custom Raft log storage and Raft RPC support, easy to integrate with latest I/O techs
  • Prometheus based health metrics support
  • Built-in tool to repair Raft clusters that permanently lost the quorum
  • Extensively tested including using Jepsen's Knossos linearizability checker, some results are here

Most features covered in Diego Ongaro's Raft thesis have been supported -

  • leader election, log replication, snapshotting and log compaction
  • membership changes
  • ReadIndex protocol for read-only queries
  • leadership transfer
  • non-voting members
  • idempotent updates transparent to applications
  • batching and pipelining
  • disk based state machine

Performance

Dragonboat is the fastest open source multi-group Raft implementation on Github.

For 3-nodes system using mid-range hardware (details here) and in-memory state machine, Dragonboat can sustain at 9 million writes per second when the payload is 16bytes each or 11 million mixed I/O per second at 9:1 read:write ratio. High throughput is maintained in geographically distributed environment. When the RTT between nodes is 30ms, 2 million I/O per second can still be achieved using a much larger number of clients. throughput

The number of concurrent active Raft groups affects the overall throughput as requests become harder to be batched. On the other hand, having thousands of idle Raft groups has a much smaller impact on throughput. nodes

Table below shows write latencies in millisecond, Dragonboat has <5ms P99 write latency when handling 8 million writes per second at 16 bytes each. Read latency is lower than writes as the ReadIndex protocol employed for linearizable reads doesn't require fsync-ed disk I/O.

Ops Payload Size 99.9% percentile 99% percentile AVG
1m 16 2.24 1.19 0.79
1m 128 11.11 1.37 0.92
1m 1024 71.61 25.91 3.75
5m 16 4.64 1.95 1.16
5m 128 36.61 6.55 1.96
8m 16 12.01 4.65 2.13

When tested on a single Raft group, Dragonboat can sustain writes at 1.25 million per second when payload is 16 bytes each, average latency is 1.3ms and the P99 latency is 2.6ms. This is achieved when using an average of 3 cores (2.8GHz) on each server.

As visualized below, Stop-the-World pauses caused by Go1.11's GC are sub-millisecond on highly loaded systems. Such very short Stop-the-World pause time is further significantly reduced in Go 1.12. Golang's runtime.ReadMemStats reports that less than 1% of the available CPU time is used by GC on highly loaded system. stw

Requirements

  • x86_64 Linux or MacOS, Go 1.13 or 1.12, GCC or Clang with C++11 support
  • RocksDB 5.13.4 or above when using RocksDB for storing Raft logs

Getting Started

Master is our unstable branch for development. Please use released versions for any production purposes.

Make sure Go 1.12 or above has been installed. Instructions below require Go module support.

You need to decide whether to use RocksDB or LevelDB to store Raft logs. RocksDB is recommended.

Install RocksDB

If RocksDB 5.13.4 or above has not already been installed, follow the steps below to install it first.

To download Dragonboat to $HOME/src and install RocksDB to /usr/local/lib and /usr/local/include:

$ cd $HOME/src
$ git clone https://github.com/lni/dragonboat
$ cd $HOME/src/dragonboat
$ make install-rocksdb-ull

Run built-in tests to check the installation:

$ cd $HOME/src/dragonboat
$ GO111MODULE=on make dragonboat-test

Once completed, $HOME/src/dragonboat can be safely deleted if you just plan to use dragonboat in your application.

Use Dragonboat

To use dragonboat in your application, make sure to import the package github.com/lni/dragonboat/v3. Also add "github.com/lni/dragonboat/v3 v3.1.3" to the require section of your project's go.mod file.

When building your application, you may need to tell Go where is the installed RocksDB library:

CGO_CFLAGS="-I/path/to/rocksdb/include" CGO_LDFLAGS="-L/path/to/rocksdb/lib -lrocksdb" go build -v pkgname

You can also follow our examples on how to use Dragonboat.

Use LevelDB

No extra dependency is required when choosing to use LevelDB based Raft log storage.

To use LevelDB based Raft log storage in your application, set the LogDBFactory field of your config.NodeHostConfig to the factory function leveldb.NewLogDB provided in the github.com/lni/dragonboat/v3/plugin/leveldb package.

To build the your application when you don't have RocksDB installed:

go build -v -tags="dragonboat_no_rocksdb" pkgname

Documents

FAQ, docs, step-by-step examples, DevOps doc, CHANGELOG and online chat are available.

C++ Binding info can be found here.

Examples

Dragonboat examples are here.

Status

Dragonboat is production ready.

Contributing

For reporting bugs, please open an issue. For contributing improvements or new features, please send in the pull request.

License

Dragonboat is licensed under the Apache License Version 2.0. See LICENSE for details.

Third party code used in Dragonboat and their licenses is summarized here.

You can’t perform that action at this time.