Priority-based semi-deterministic leader election.

[zongtanghu]

Motivation:

The main goal is to prioritize member nodes for next leader election. Raft uses randomized timer so that it is hard to make a deterministic decision. In this library, leader election will be still probabilistic, but more predictable.

Modification:

The main modification of codes is below:
(1)NodeImpl.java;
(2)Node.java;
(3)PeerId.java;
(4)NodeOptions.java;
(5)NodeTest.java;
(6)PeerIdTest.java;
(7)TestUtils.java;
(8)TestCluster.java;
(9)NodeTest;

Result:

Fixes ISSUE #264

If there is no issue then describe the changes introduced by this PR.

[zongtanghu]

Hi @fengjiachun @killme2008 ,I have already implemented the initial version of priority-based semi-deterministic leader election.I'm adjusting many test cases in my local enviroment,because this pr refers to many test cases.You can review this pr firstly.And it will be much more efficient if we are reviewing pr's codes and adjusting codes and test cases at the same time.

[masaimu]

提了些问题

[killme2008]

Cool, i will review it ASAP.

[zongtanghu]

Cool, i will review it ASAP.

I have already adjust some parts of codes in my local branch and will push again tonight. @killme2008 @masaimu @fengjiachun

[masaimu]

提了几个问题，可以参考

[zongtanghu]

Hi @killme2008 @fengjiachun @masaimu ,I have take the details of priority election into account again.I agree with @fengjiachun .The progress that If candidate_priority < target_priority, it rejects the vote request should be deperated.

[fengjiachun]

应该可以不改原有单测， start 增加一个方法，原有方法保持默认 disabled 即可吧？
改了单测我很难判断和老版本的兼容性

[fengjiachun]

同上，下面不再重复

[fengjiachun]

需要 if else ？ priority 直接传进去就好

[zongtanghu]

Okay

[killme2008]

这里我比较建议加个布尔值参数， inLock，如果是 false，才去加这个 writeLock，很多调用的地方可能已经在锁里了。

[zongtanghu]

Okay

[killme2008]

maxPriority初始值还是用当前 node 的 priority 吧，看起来更合理，虽然结果没有区别。

[fengjiachun]

这个原来是用当前 node 的 priority 的，是我给了建议改成 Integer.MIN_VALUE， 主要原因是 Node 还有一种 learner 角色，是不包含在入参 peerIds nodes 中的，所以在这里使用 learner node 的 priority 还是有些奇怪的，虽然结果都没区别

[killme2008]

其他没有太多问题了，可以考虑合并 @fengjiachun

[killme2008]

测试有奇怪错误：

41.92s$ mvn clean install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
0.04s$ sh ./tools/check_format.sh
Please commit your change before run this shell, un commit files:
 M jraft-core/src/main/java/com/alipay/sofa/jraft/core/NodeImpl.java
./tools/check_format.sh: 7: exit: Illegal number: -1
The command "sh ./tools/check_format.sh" failed and exited with 2 during .

[zongtanghu]

这个问题的话，是我commit代码时候，没有本地install下，所以code formats有一些不正确；我来调整下 huzongtang@cmss.chinamobile.com From: dennis zhuang Date: 2019-11-15 15:04 To: sofastack/sofa-jraft CC: Hu Zongtang; Author Subject: Re: [sofastack/sofa-jraft] Priority-based semi-deterministic leader election. (#334) 测试有奇怪错误： 41.92s$ mvn clean install -DskipTests=true -Dmaven.javadoc.skip=true -B -V 0.04s$ sh ./tools/check_format.sh Please commit your change before run this shell, un commit files: M jraft-core/src/main/java/com/alipay/sofa/jraft/core/NodeImpl.java ./tools/check_format.sh: 7: exit: Illegal number: -1 The command "sh ./tools/check_format.sh" failed and exited with 2 during . — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[zongtanghu]

Okay. huzongtang@cmss.chinamobile.com From: dennis zhuang Date: 2019-11-15 15:00 To: sofastack/sofa-jraft CC: Hu Zongtang; Author Subject: Re: [sofastack/sofa-jraft] Priority-based semi-deterministic leader election. (#334) @killme2008 commented on this pull request. In jraft-core/src/main/java/com/alipay/sofa/jraft/core/NodeImpl.java:

+ // Default Gap value should be bigger than 10.

+ final int decayPriorityGap = Math.max(this.options.getDecayPriorityGap(), 10); + final int gap = Math.max(decayPriorityGap, (this.targetPriority / 5)); + + final int prevTargetPriority = this.targetPriority; + this.targetPriority = Math.max(ElectionPriority.MinValue, (this.targetPriority - gap)); + LOG.info("Node {} priority decay, from: {}, to: {}.", getNodeId(), prevTargetPriority, this.targetPriority); + } + + /** + * Check and set configuration for node.At the same time, if configuration is changed, + * then compute and update the target priority value. + * + */ + private void checkAndSetConfiguration() { + this.writeLock.lock(); 这里我比较建议加个布尔值参数， inLock，如果是 false，才去加这个 writeLock，很多调用的地方可能已经在锁里了。 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[zongtanghu]

Hi 伯岩，但是如果考虑到之前learner的特性，如果这块逻辑再learner上执行，maxPriority初始值的serverId.getPriority()，其实也是不对的，所以说maxPriority值与node无关，只与peerIds有关 huzongtang@cmss.chinamobile.com From: dennis zhuang Date: 2019-11-15 15:00 To: sofastack/sofa-jraft CC: Hu Zongtang; Author Subject: Re: [sofastack/sofa-jraft] Priority-based semi-deterministic leader election. (#334) @killme2008 commented on this pull request. In jraft-core/src/main/java/com/alipay/sofa/jraft/core/NodeImpl.java:

+ }

+ } finally { + this.writeLock.unlock(); + } + } + + /** + * Get max priority value for all nodes in the same Raft group, and update current node's target priority value. + * + * @param peerIds peer nodes in the same Raft group + * + */ + private int getMaxPriorityOfNodes(final List<PeerId> peerIds) { + Requires.requireNonNull(peerIds, "Null peer list"); + + int maxPriority = Integer.MIN_VALUE; maxPriority初始值还是用当前 node 的 priority 吧，看起来更合理，虽然结果没有区别。 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[fengjiachun]

其他没有太多问题了，可以考虑合并 @fengjiachun

这几天我会做一次 jepsen 验证，通过后合并