Skip to content

Commit

Permalink
dragonboat: enabled Go module support
Browse files Browse the repository at this point in the history
  • Loading branch information
lni committed Jun 21, 2019
1 parent 75e9b0d commit 95e86bd
Show file tree
Hide file tree
Showing 199 changed files with 675 additions and 663 deletions.
7 changes: 4 additions & 3 deletions CHANGELOG.md
@@ -1,4 +1,4 @@
## v3.0 (Release date TBD)
## v3.0 (2019-06-21)

Dragonboat v3.0 is a major release with breaking changes. Please make sure to carefully read the CHANGELOG below before upgrading.

Expand All @@ -9,6 +9,7 @@ Dragonboat v3.0 is a major release with breaking changes. Please make sure to ca
- Added the ability to use exported snapshot to repair permanently damaged cluster that no longer has majority quorum.
- Added new API for cleaning up data and release disk spaces after a node is removed from its Raft cluster.
- Added the ability to limit peak memory usage when disk or network is slow.
- Added Go module support. Go 1.12 is required.

### Improvements

Expand All @@ -21,12 +22,12 @@ Dragonboat v3.0 is a major release with breaking changes. Please make sure to ca
- The Drummer package has been made invisible from user applications.
- The statemachine.IStateMachine interface has been upgraded to reflect the fact that not all state machine data is stored in memory ([#46](https://github.com/lni/dragonboat/issues/46)).

## v2.1 (Feb 2019)
## v2.1 (2019-02-20)

### New features

- Added support to store Raft Logs in LevelDB.

## v2.0 (Jan 2019)
## v2.0 (2019-01-04)

Initial open source release.
3 changes: 2 additions & 1 deletion Makefile
Expand Up @@ -15,6 +15,8 @@
OS := $(shell uname)
# the location of this Makefile
PKGROOT=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
# name of the package
PKGNAME=github.com/lni/dragonboat/v3
# set the environmental variable DRAGONBOAT_LOGDB to lmdb to use lmdb based
# LogDB implementation.
ifeq ($(DRAGONBOAT_LOGDB),leveldb)
Expand Down Expand Up @@ -125,7 +127,6 @@ SELECTED_BENCH_OPTION=-run ^$$ -bench=.
endif

INSTALL_PATH ?= /usr/local
PKGNAME=github.com/lni/dragonboat
# shared lib and version number related
LIBNAME=libdragonboat
PLATFORM_SHARED_EXT=so
Expand Down
52 changes: 27 additions & 25 deletions README.CHS.md
Expand Up @@ -8,8 +8,8 @@
[![Join the chat at https://gitter.im/lni/dragonboat](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/lni/dragonboat?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

## 项目新闻 ##
* 2019-06-09 Dragonboat v3.0 测试版([CHANGELOG](CHANGELOG.md))被合并到了Master branch中
* 2019-02-20 Dragonboat v2.0 已发布。
* 2019-06-21 Dragonboat v3.0 已发布,新增基于磁盘的状态机和Go module支持,详见([CHANGELOG](CHANGELOG.md))。
* 2019-02-20 Dragonboat v2.1 已发布。

## 关于 ##
Dragonboat是一个高性能[Go](https://golang.org)实现的多组[Raft](https://raft.github.io/) [共识算法](https://en.wikipedia.org/wiki/Consensus_(computer_science))库,它同时提供[C++11](/binding)支持。
Expand Down Expand Up @@ -63,44 +63,46 @@ Dragonboat是目前Github网站上最快的开源多组Raft实现。
![stw](./docs/stw.png)

## 系统需求 ##
* x86_64 Linux或MacOS, Go 1.11, 1.12,支持C++11的近期版本GCC或Clang
* x86_64 Linux或MacOS, Go 1.12,支持C++11的近期版本GCC或Clang
* [RocksDB](https://github.com/facebook/rocksdb/blob/master/INSTALL.md) 5.13.4或更新的版本

## 开始使用 ##
__上列步骤使用的是Master branch的代码。Master是用于开发的非稳定branch。生产环境请使用已发布版本。__
__Master是用于开发的非稳定branch。生产环境请使用已发布版本。__

下载Dragonboat库至您的[Go工作环境](https://golang.org/doc/install):
```
$ go get -u -d github.com/lni/dragonboat
```
请选择使用[RocksDB还是LevelDB](docs/storage.CHS.md)来存储Raft日志数据,建议使用RocksDB。
首先请确保Go 1.12或者更新的版本已被安装以获得[Go module](https://github.com/golang/go/wiki/Modules)支持。

请首先选择使用[RocksDB还是LevelDB](docs/storage.CHS.md)来存储Raft日志数据,建议使用RocksDB。

### RocksDB ###
如果RocksDB 5.13.4或者更新版本尚未安装:
### 安装RocksDB ###
如果RocksDB 5.13.4或者更新版本尚未安装,可按下列步骤安装。首先下载Dragonboat库至$HOME/src并将RocksDB安装到/usr/local/lib和/usr/local/include位置
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ cd $HOME/src
$ git clone https://github.com/lni/dragonboat
$ cd $HOME/src/dragonboat
$ make install-rocksdb-ull
```
上述命令将把RocksDB 5.13.4安装到/usr/local/lib和/usr/local/include/rocksdb。如果RocksDB已经被安装则可跳过此步。

运行内建测试以检查安装是否完成:
运行下列命令检查安装是否正确:
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ make dragonboat-test
$ cd $HOME/src/dragonboat
$ GO111MODULE=on make dragonboat-test
```
编译您自己的基于Dragonboat的应用:

请注意,如果RocksDB事先已经安装,上述步骤可直接跳过。如果您仅希望使用dragonboat库,至此可以安全的删除$HOME/src/dragonboat目录了。

### 使用Dragonboat ###
在您的应用中使用dragonboat库,请确保在Go程序代码中import __github.com/lni/dragonboat/v3__这个包,同时把"github.com/lni/dragonboat/v3 v3.0.0"添加到您的Go应用的go.mod文件的__require__部分。

编译您的应用的时候,如有需要,可将RocksDB安装位置告知Go:
```
CGO_CFLAGS="-I/path/to/rocksdb/include" CGO_LDFLAGS="-L/path/to/rocksdb/lib -lrocksdb" go build -v pkgname
```

具体使用可可参考[示例](https://github.com/lni/dragonboat-example)

### LevelDB ###
使用LevelDB无额外安装步骤,运行上述同样的测试:
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ DRAGONBOAT_LOGDB=leveldb make dragonboat-test
```
在应用中使用基于LevelDB的Raft log storage,需将您的config.NodeHostConfig的LogDBFactory项设为leveldb.NewLogDB这一在github.com/lni/dragonboat/plugin/leveldb包中提供的factory函数。
使用LevelDB无额外安装步骤。在应用中使用基于LevelDB的Raft log storage,需将您的config.NodeHostConfig的LogDBFactory项设为leveldb.NewLogDB这一在github.com/lni/dragonboat/plugin/leveldb包中提供的factory函数。

编译应用时可如此避免对RocksDB库的依赖:
编译应用时可用如下方法避免对RocksDB库的依赖:
```
go build -v -tags="dragonboat_no_rocksdb" pkgname
```
Expand Down
53 changes: 28 additions & 25 deletions README.md
Expand Up @@ -8,8 +8,8 @@
[![Join the chat at https://gitter.im/lni/dragonboat](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/lni/dragonboat?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

## News ##
* 2019-06-09 Dragonboat v3.0 beta ([CHANGELOG](CHANGELOG.md)) is now available in the Master branch.
* 2019-02-20 Dragonboat v2.0 has been released.
* 2019-06-21 Dragonboat v3.0 has been released with on disk state machine and Go module support ([CHANGELOG](CHANGELOG.md)).
* 2019-02-20 Dragonboat v2.1 has been released.

## About ##
Dragonboat is a high performance multi-group [Raft](https://raft.github.io/) [consensus](https://en.wikipedia.org/wiki/Consensus_(computer_science)) library in [Go](https://golang.org/) with [C++11 binding](/binding) support.
Expand Down Expand Up @@ -62,50 +62,53 @@ As visualized below, Stop-the-World pauses caused by Go1.11's GC are sub-millise
![stw](./docs/stw.png)

## Requirements ##
* x86_64 Linux or MacOS, Go 1.11 or 1.12, GCC or Clang with C++11 support
* x86_64 Linux or MacOS, Go 1.12, GCC or Clang with C++11 support
* [RocksDB](https://github.com/facebook/rocksdb/blob/master/INSTALL.md) 5.13.4 or above when using RocksDB for storing Raft logs

## Getting Started ##
__Note that, steps below use code from the Master branch. Master is our unstable branch for development. Please use released versions for any production purposes.__
__Master is our unstable branch for development. Please use released versions for any production purposes.__

To download Dragonboat to your [Go workspace](https://golang.org/doc/install):
```
$ go get -u -d github.com/lni/dragonboat
```
You need to decide whether to use [RocksDB or LevelDB](docs/storage.md) to store Raft logs. RocksDB is recommended.
Make sure Go 1.12 or above has been installed. Instructions below require [Go module](https://github.com/golang/go/wiki/Modules) support.

You need to decide whether to use [RocksDB or LevelDB](docs/storage.md) to store Raft logs. RocksDB is recommended.

### Install RocksDB ###
If RocksDB 5.13.4 or above has not already been installed, follow the steps below to install it first.

### RocksDB ###
If RocksDB 5.13.4 or above has not been installed, use the following commands to install RocksDB 5.13.4 to /usr/local/lib and /usr/local/include.
To download Dragonboat to $HOME/src and install RocksDB to /usr/local/lib and /usr/local/include:
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ cd $HOME/src
$ git clone https://github.com/lni/dragonboat
$ cd $HOME/src/dragonboat
$ make install-rocksdb-ull
```
Run built-in tests to check the installation:
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ make dragonboat-test
$ cd $HOME/src/dragonboat
$ GO111MODULE=on make dragonboat-test
```
To build your application
Once completed, $HOME/src/dragonboat can be safely deleted if you just plan to use dragonboat in your application.

### Use Dragonboat ###
To use dragonboat in your application, make sure to import the package __github.com/lni/dragonboat/v3__ in your Go code. Also add "github.com/lni/dragonboat/v3 v3.0.0" to the __require__ section of your go.mod file.

When building your application, you may need to tell Go where is the installed RocksDB library:
```
CGO_CFLAGS="-I/path/to/rocksdb/include" CGO_LDFLAGS="-L/path/to/rocksdb/lib -lrocksdb" go build -v pkgname
```
You can also follow our [examples](https://github.com/lni/dragonboat-example) on how to use Dragonboat.

### LevelDB ###
Nothing need to be installed when using LevelDB based Raft Log storage.
### Use LevelDB ###
No extra dependency is required when choosing to use LevelDB based Raft log storage.

To run built-in tests using LevelDB based storage:
```
$ cd $GOPATH/src/github.com/lni/dragonboat
$ DRAGONBOAT_LOGDB=leveldb make dragonboat-test
```
To use LevelDB based Raft log storage in your application, set the LogDBFactory field of your config.NodeHostConfig to the factory function leveldb.NewLogDB provided in the github.com/lni/dragonboat/plugin/leveldb package.
To use LevelDB based Raft log storage in your application, set the LogDBFactory field of your config.NodeHostConfig to the factory function leveldb.NewLogDB provided in the github.com/lni/dragonboat/v3/plugin/leveldb package.

To build the your application when you don't have RocksDB installed
To build the your application when you don't have RocksDB installed:
```
go build -v -tags="dragonboat_no_rocksdb" pkgname
```

### Documents ###
## Documents ##
[FAQ](https://github.com/lni/dragonboat/wiki/FAQ), [docs](https://godoc.org/github.com/lni/dragonboat), step-by-step [examples](https://github.com/lni/dragonboat-example), [DevOps doc](docs/devops.md), [CHANGELOG](CHANGELOG.md) and [online chat](https://gitter.im/lni/dragonboat) are available.

C++ Binding info can be found [here](https://github.com/lni/dragonboat/blob/master/binding/README.md).
Expand Down
22 changes: 11 additions & 11 deletions benchmark_test.go
Expand Up @@ -25,17 +25,17 @@ import (
"testing"
"time"

"github.com/lni/dragonboat/client"
"github.com/lni/dragonboat/config"
"github.com/lni/dragonboat/internal/rsm"
"github.com/lni/dragonboat/internal/server"
"github.com/lni/dragonboat/internal/settings"
"github.com/lni/dragonboat/internal/tests"
"github.com/lni/dragonboat/internal/transport"
"github.com/lni/dragonboat/internal/utils/random"
"github.com/lni/dragonboat/logger"
pb "github.com/lni/dragonboat/raftpb"
sm "github.com/lni/dragonboat/statemachine"
"github.com/lni/dragonboat/v3/client"
"github.com/lni/dragonboat/v3/config"
"github.com/lni/dragonboat/v3/internal/rsm"
"github.com/lni/dragonboat/v3/internal/server"
"github.com/lni/dragonboat/v3/internal/settings"
"github.com/lni/dragonboat/v3/internal/tests"
"github.com/lni/dragonboat/v3/internal/transport"
"github.com/lni/dragonboat/v3/internal/utils/random"
"github.com/lni/dragonboat/v3/logger"
pb "github.com/lni/dragonboat/v3/raftpb"
sm "github.com/lni/dragonboat/v3/statemachine"
)

func benchmarkNoPool128Allocs(b *testing.B, sz uint64) {
Expand Down
12 changes: 6 additions & 6 deletions binding.go
Expand Up @@ -20,12 +20,12 @@ import (
"time"
"unsafe"

"github.com/lni/dragonboat/client"
"github.com/lni/dragonboat/config"
"github.com/lni/dragonboat/internal/cpp"
"github.com/lni/dragonboat/internal/rsm"
"github.com/lni/dragonboat/internal/utils/fileutil"
pb "github.com/lni/dragonboat/raftpb"
"github.com/lni/dragonboat/v3/client"
"github.com/lni/dragonboat/v3/config"
"github.com/lni/dragonboat/v3/internal/cpp"
"github.com/lni/dragonboat/v3/internal/rsm"
"github.com/lni/dragonboat/v3/internal/utils/fileutil"
pb "github.com/lni/dragonboat/v3/raftpb"
)

//
Expand Down
16 changes: 8 additions & 8 deletions binding/binding.go
Expand Up @@ -34,14 +34,14 @@ import (
"time"
"unsafe"

"github.com/lni/dragonboat"
"github.com/lni/dragonboat/client"
"github.com/lni/dragonboat/config"
"github.com/lni/dragonboat/internal/cpp"
"github.com/lni/dragonboat/internal/utils/leaktest"
"github.com/lni/dragonboat/internal/utils/random"
"github.com/lni/dragonboat/internal/utils/syncutil"
"github.com/lni/dragonboat/logger"
"github.com/lni/dragonboat/v3"
"github.com/lni/dragonboat/v3/client"
"github.com/lni/dragonboat/v3/config"
"github.com/lni/dragonboat/v3/internal/cpp"
"github.com/lni/dragonboat/v3/internal/utils/leaktest"
"github.com/lni/dragonboat/v3/internal/utils/random"
"github.com/lni/dragonboat/v3/internal/utils/syncutil"
"github.com/lni/dragonboat/v3/logger"
)

func init() {
Expand Down
2 changes: 1 addition & 1 deletion binding/types.go
Expand Up @@ -21,7 +21,7 @@ import (
"reflect"
"unsafe"

"github.com/lni/dragonboat"
"github.com/lni/dragonboat/v3"
)

func charToByte(data *C.char, len C.size_t) []byte {
Expand Down
2 changes: 1 addition & 1 deletion client/session.go
Expand Up @@ -17,7 +17,7 @@ package client
import (
"math"

"github.com/lni/dragonboat/internal/utils/random"
"github.com/lni/dragonboat/v3/internal/utils/random"
)

const (
Expand Down
2 changes: 1 addition & 1 deletion client/session_test.go
Expand Up @@ -17,7 +17,7 @@ package client
import (
"testing"

"github.com/lni/dragonboat/internal/utils/random"
"github.com/lni/dragonboat/v3/internal/utils/random"
)

func TestNoOPSessionHasExpectedSeriesID(t *testing.T) {
Expand Down
12 changes: 6 additions & 6 deletions config/config.go
Expand Up @@ -22,11 +22,11 @@ import (
"crypto/tls"
"errors"

"github.com/lni/dragonboat/internal/settings"
"github.com/lni/dragonboat/internal/utils/netutil"
"github.com/lni/dragonboat/internal/utils/stringutil"
"github.com/lni/dragonboat/logger"
"github.com/lni/dragonboat/raftio"
"github.com/lni/dragonboat/v3/internal/settings"
"github.com/lni/dragonboat/v3/internal/utils/netutil"
"github.com/lni/dragonboat/v3/internal/utils/stringutil"
"github.com/lni/dragonboat/v3/logger"
"github.com/lni/dragonboat/v3/raftio"
)

var (
Expand Down Expand Up @@ -200,7 +200,7 @@ type NodeHostConfig struct {
// MutualTLS defines whether to use mutual TLS for authenticating servers
// and clients. Insecure communication is used when MutualTLS is set to
// False.
// See https://github.com/lni/dragonboat/wiki/TLS-in-Dragonboat for more
// See https://github.com/lni/dragonboat/v3/wiki/TLS-in-Dragonboat for more
// details on how to use Mutual TLS.
MutualTLS bool
// CAFile is the path of the CA certificate file. This field is ignored when
Expand Down
2 changes: 1 addition & 1 deletion config/config_test.go
Expand Up @@ -19,7 +19,7 @@ import (
)

func ExampleNodeHostConfig() {
// import "github.com/lni/dragonboat/plugin/rpc"
// import "github.com/lni/dragonboat/v3/plugin/rpc"
nhc := NodeHostConfig{
WALDir: "/data/wal",
NodeHostDir: "/data/dragonboat-data",
Expand Down
6 changes: 3 additions & 3 deletions example_test.go
Expand Up @@ -22,9 +22,9 @@ import (
"log"
"time"

"github.com/lni/dragonboat/config"
"github.com/lni/dragonboat/internal/tests"
sm "github.com/lni/dragonboat/statemachine"
"github.com/lni/dragonboat/v3/config"
"github.com/lni/dragonboat/v3/internal/tests"
sm "github.com/lni/dragonboat/v3/statemachine"
)

var enh *NodeHost
Expand Down
16 changes: 8 additions & 8 deletions execengine.go
Expand Up @@ -18,14 +18,14 @@ import (
"sync"
"time"

"github.com/lni/dragonboat/internal/rsm"
"github.com/lni/dragonboat/internal/server"
"github.com/lni/dragonboat/internal/settings"
"github.com/lni/dragonboat/internal/tests"
"github.com/lni/dragonboat/internal/utils/syncutil"
"github.com/lni/dragonboat/raftio"
pb "github.com/lni/dragonboat/raftpb"
sm "github.com/lni/dragonboat/statemachine"
"github.com/lni/dragonboat/v3/internal/rsm"
"github.com/lni/dragonboat/v3/internal/server"
"github.com/lni/dragonboat/v3/internal/settings"
"github.com/lni/dragonboat/v3/internal/tests"
"github.com/lni/dragonboat/v3/internal/utils/syncutil"
"github.com/lni/dragonboat/v3/raftio"
pb "github.com/lni/dragonboat/v3/raftpb"
sm "github.com/lni/dragonboat/v3/statemachine"
)

var (
Expand Down
2 changes: 1 addition & 1 deletion execengine_test.go
Expand Up @@ -20,7 +20,7 @@ package dragonboat
import (
"testing"

"github.com/lni/dragonboat/internal/rsm"
"github.com/lni/dragonboat/v3/internal/rsm"
)

func TestWorkReadyCanBeCreated(t *testing.T) {
Expand Down

0 comments on commit 95e86bd

Please sign in to comment.