Quorum raft upgrade v3.3.13 #887

vsmk98 · 2019-11-19T13:37:53Z

This PR contains the following changes to Raft consensus:

Upgrade quorum's Raft library to version v3.3.13 from upstream.
Raft API to add raft learner node in quorum. learner node will follow the leader but will not vote or take part in quorum for commits.
Raft API for promoting a learner node to peer node
Disallow learner node from adding another learner or peer
Disallow learner node from promoting itself or other learners to peer
Disallow learner node to remove other peers or learners from raft cluster
Allow learner node to remove itself from raft cluster

#825, #790, #852, #180

fix learner node changing to voter node when network restarted

…d-upgrade # Conflicts: # vendor/github.com/coreos/etcd/raft/raft.go

add --raftlearner flag to geth to specify node type when joining

remove debug log statements

…ly when it joins cluster first time and remove my fixes for the same

…becoming a learner

…peer

…ate api

…pgrade # Conflicts: # raft/handler.go

…during snapshot restore

….3.13) package. This is based on the same convention followed by previous version(3.1.0)

allow raft learner node to call removePeer if raftId to be removed is its own raftId update raft.cluster api to show minter instead of leader update raft consensus documentation code improvements

…-upgrade-v3.3.13 # Conflicts: # raft/handler.go # raft/peer.go

lookfwd · 2019-11-22T06:48:15Z

Wondering how has this been tested?

…-upgrade-v3.3.13

amalrajmani · 2019-11-25T06:11:52Z

Wondering how has this been tested?

When a new node joins the network as a peer, it tries to sync with leader and during this time the leader is overloaded and can potentially lead to periods of cluster unavailability. Please refer to https://github.com/etcd-io/etcd/blob/master/Documentation/learning/design-learner.md for further details on various scenarios. The advantage of a learner node is it joins the network as a non voting member and hence does not impact the Raft quorum.
For testing this, brought up a 3 node raft network and created few thousand blocks. Then added a 4th node as peer. While the 4th node is syncing with the network, brought down one of the other nodes. Now if any transaction is send, the transaction will wait till the newly joined node is fully synced with the network.
Repeated the same test adding the 4th node as learner. In this case the network will be able to process the transactions while the new learner node is still syncing with the network

jbhurat

When a node is learner and it calls raft.removePeer() on itself, it returns null. When checking raft.cluster the node is still seen with a verifier role. Running raft.role returns an empty string. Running raft.cluster from any other node returns correct results though

…at removed itself from raft cluster

… when restores snapshot'

jbhurat

LGTM

amalrajmani · 2019-11-27T08:14:05Z

When a node is learner and it calls raft.removePeer() on itself, it returns null. When checking raft.cluster the node is still seen with a verifier role. Running raft.role returns an empty string. Running raft.cluster from any other node returns correct results though

this has been fixed now. after a learner/peer node removes itself it will return an empty array when raft.cluster is called

1. let raft step return error when proposal is dropped to allow fail-fast 2. Propose in raft node wait the proposal result so we can fail fast while dropping proposal

trung

LGTM. Changes in the original etcd raft library:

Inject a channel to capture role change: roleC
Populate ConfState to separate voters and learners, similar to raft: raft learners should be returned after applyConfChange by absolute8511 · Pull Request #9116 · etcd-io/etcd

amalraj.manigmail.com and others added 30 commits June 6, 2019 09:37

raft: upgrade etcd to version 3.3.13

e51f398

raft: upgrade etcd to version 3.3.13

2c73795

raft: upgrade etcd to version 3.3.13 - add vendor dependencies

992e850

raft: upgrade etcd to version 3.3.13 - add vendor dependencies

ef06c2b

raft: upgrade etcd to version 3.3.13 - add vendor dependencies

50cb644

raft: upgrade etcd to version 3.3.13 - remove unused code

2b43dd5

raft: added api for adding a learner node

d72bc7b

raft: added api for adding a learner node

1cfced8

raft: added api for adding a learner node

db92e28

raft: added api for adding a learner node

419269f

raft: add isLearner flag to raft's peer Address

0c14dbe

fix learner node changing to voter node when network restarted

Merge remote-tracking branch 'quoeng/raft-etcd-upgrade' into raft-etc…

d4304d1

…d-upgrade # Conflicts: # vendor/github.com/coreos/etcd/raft/raft.go

raft: add api to promote learner node to voter node

48421d4

raft: fix start node to handle starting learner node to join cluster

6421692

add --raftlearner flag to geth to specify node type when joining

raft: remove is learner flag from raft.address

3dd6e6c

raft: add raft.confState api to view confState

544464a

remove debug log statements

raft: code clean up and improvements

99ac09f

raft: code clean up and improvements

1532ec1

raft: apply fix from raft upstream to allow learner to restore proper…

5e18899

…ly when it joins cluster first time and remove my fixes for the same

add raft test case

a0f4db5

raft: update fix from upstream raft library to stop normal peer from …

c23ee79

…becoming a learner

raft: don't allow learner node to add learner/peer or remove/promote …

a161d37

…peer

raft: code improvement to conf change add node/learner handling

d5c169c

raft: code improvement to conf change add node/learner handling

38d6a59

raft: remove incomplete test

8a05a70

raft: enhance raft.cluster api to display role and remove raft.confSt…

c22043a

…ate api

raft: format code

c9a4960

raft: go test for add learner and promote learner to peer

66231eb

raft: go test code improvements

674f85d

raft: go test code improvements

4552a73

amalraj.manigmail.com and others added 7 commits September 29, 2019 20:00

Merge remote-tracking branch 'remotes/origin/master' into raft-etcd-u…

1c7c165

…pgrade # Conflicts: # raft/handler.go

raft: minor code improvements based on review comments

cbaa06e

raft: update upstream fix for preventing voter from becoming learner …

3e3891a

…during snapshot restore

raft: remove unused packages and go test files from the coreos/etcd(3…

4e59b83

….3.13) package. This is based on the same convention followed by previous version(3.1.0)

raft review feedback:

ddcd689

allow raft learner node to call removePeer if raftId to be removed is its own raftId update raft.cluster api to show minter instead of leader update raft consensus documentation code improvements

Merge remote-tracking branch 'remotes/origin/master' into Quroum-raft…

160bcf6

…-upgrade-v3.3.13 # Conflicts: # raft/handler.go # raft/peer.go

update to documentation

e32d1f1

vsmk98 requested review from jbhurat and jpmsam November 19, 2019 13:38

raft: method rename - based on review feedback

7c8fe87

Merge remote-tracking branch 'remotes/origin/master' into Quorum-raft…

8a521c2

…-upgrade-v3.3.13

jbhurat reviewed Nov 25, 2019

View reviewed changes

amalraj.manigmail.com added 2 commits November 26, 2019 17:54

raft: return empty cluster when raft.cluster api is called on node th…

5562535

…at removed itself from raft cluster

raft: take upstream change to remove validation 'can't become learner…

717a744

… when restores snapshot'

jbhurat previously approved these changes Nov 26, 2019

View reviewed changes

jpmsam requested a review from trung November 29, 2019 19:15

Merge branch 'master' into Quorum-raft-upgrade-v3.3.13

65d7c43

zzy96 dismissed jbhurat’s stale review via 65d7c43 December 2, 2019 14:57

zzy96 and others added 2 commits December 2, 2019 22:58

Merge branch 'master' into Quorum-raft-upgrade-v3.3.13

4d0c74f

Cherry picking changes from etcd master for

9412d53

1. let raft step return error when proposal is dropped to allow fail-fast 2. Propose in raft node wait the proposal result so we can fail fast while dropping proposal

trung approved these changes Dec 3, 2019

View reviewed changes

Merge branch 'master' into Quorum-raft-upgrade-v3.3.13

744b34d

jpmsam approved these changes Dec 3, 2019

View reviewed changes

jpmsam merged commit dfd93ff into master Dec 4, 2019

trung deleted the Quorum-raft-upgrade-v3.3.13 branch December 6, 2019 14:09

easeev mentioned this pull request Dec 23, 2019

Add learner nodes and hostname support chainstack/web3quorum#1

Merged

stu4515 mentioned this pull request Jul 5, 2022

Add learner node synechron-finlabs/quorum-maker#140

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Quorum raft upgrade v3.3.13 #887

Quorum raft upgrade v3.3.13 #887

vsmk98 commented Nov 19, 2019 •

edited

lookfwd commented Nov 22, 2019

amalrajmani commented Nov 25, 2019

jbhurat left a comment

jbhurat left a comment

amalrajmani commented Nov 27, 2019

trung left a comment

Quorum raft upgrade v3.3.13 #887

Quorum raft upgrade v3.3.13 #887

Conversation

vsmk98 commented Nov 19, 2019 • edited

lookfwd commented Nov 22, 2019

amalrajmani commented Nov 25, 2019

jbhurat left a comment

Choose a reason for hiding this comment

jbhurat left a comment

Choose a reason for hiding this comment

amalrajmani commented Nov 27, 2019

trung left a comment

Choose a reason for hiding this comment

vsmk98 commented Nov 19, 2019 •

edited