Replies: 7 comments 8 replies
-
Although the guarantees on safety and liveness are defined by the number of faulty replicas, the performance of MinBFT seems to be guaranteed to be best under more struct condition that the network is stable enough. My point is that if the network is slow/unstable, the MinBFT cluster does not perform well whether view change happens frequently or not, so optimizing view-change overhead might not help us solve the situation alone. So keeping acceptance quorum minimum (option 1) makes sense most to me (for now).
Maybe we can reduce the size of ViewChange message by controlling checkpoint to keep message log small, although checkpoint is not available now. So will this be solved eventually? |
Beta Was this translation helpful? Give feedback.
-
I completely agree with Naoya's comment. It is my understanding that we can expect replicas to work normally in most cases (i.e. a view change does not occur frequently), so it's reasonable for me to choose 1, which is optimal for such situation. |
Beta Was this translation helpful? Give feedback.
-
@nhoriguchi @ynamiki Thanks for your feedback! In fact, the testing code in #214 assumes the option 1 with the new-view certificate quorum size equal to n-f. On a second thought, it is worth mentioning that there might not be any notable difference among the options in a stable system when no timeout occurs and at most f replicas are faulty or significantly slower. This is because all replicas anyway follow all-to-all communication pattern. The option 1 would only improve performance if there are more than f replicas which are significantly slower. Nevertheless, it would still be able to mask up to n-f-1 faulty replicas in a stable view (which may be more than f strictly required by the protocol guarantees). Moreover, with the option 1, if more than f replicas are significantly slower and become more and more delayed, they might eventually time out and trigger view change, thus unnecessarily interrupting the stable view. This would less likely happen with the option 2. We could try to overcome this by increasing the size of the view-change certificate (a collection of ReqViewChange messages) to match the new-view certificate size. |
Beta Was this translation helpful? Give feedback.
-
Apparently, the paper refers to the "acceptance quorum" as "commit certificate", so I adjusted the discussion's subject. |
Beta Was this translation helpful? Give feedback.
-
According to my current understanding, if all correct replicas are equally fast then, in any case, there should be no notable performance difference in a stable view. However, smaller commit certificate size may allow the system to temporarily mask more than f faulty backup replicas, though this cannot be guaranteed and users should not rely on this. On the other hand, it determines how many replicas may lag behind: up to n-|Q| replicas can be permanently delayed, where |Q| is the commit certificate size. Considering potential user's intuition about semantics of the protocol parameters, should we allow more than f correct replicas to permanently lag behind? |
Beta Was this translation helpful? Give feedback.
-
The quorum size must be at least
May I ask what is the intension of having more nodes than necessary? That is, choosing
The quorum size does not influence message sending process in stable view as you said, but it will influence the latency of reaching consensus, as a node returns earlier upon a smaller quorum of COMMIT messages. I think in a network with different connection latencies, a smaller quorum keeps the closely connected nodes moving forward faster than the rest, while a bigger-sized quorum tries to keep more nodes synchronised; this also means view-change may be triggered less likely. Depending on the network connectivity, if all nodes are more or less well synced, then i will go for a smaller quorum. |
Beta Was this translation helpful? Give feedback.
-
An important outcome from this discussion for me was that not only any new-view certificate must intersect with any commit certificate, but also that any view-change certificate (a collection of ReqViewChange messages) must intersect with any commit certificate. Otherwise, slower replicas could keep triggering view change. The changes in #222 already take this into account. |
Beta Was this translation helpful? Give feedback.
-
Hi, I would like to hear your opinion about the following.
There are two important kinds of quorums that ensure safety and liveness properties in MinBFT. Those are the acceptance quorum and the new-view certificate. The acceptance quorum is a collection of Prepare/Commit messages that is sufficient for request execution. The new-view certificate is a collection of ViewChange messages in a NewView message that is sufficient to justify a transition into a new view.
To ensure safety, any acceptance quorum must intersect with any new-view certificate in at least one replica. The intersection guarantees that any potentially executed request will propagate to the new view, even if the new primary is Byzantine. Moreover, to ensure liveness, the maximum quorum size cannot exceed n-f, since up to f replicas may be faulty and do not generate any message. On the other hand, for the sake of safety, the minimum quorum size cannot be less than f+1, since it must include at least one correct replica.
In summary, the following conditions must be satisfied:
The paper only discusses the case when n=2f+1. In that case, the only possible choice in the range of (f, n-f] is f+1, which also satisfies the condition 3. However, in the generic case when n>2f, one could consider the following options:
There is no difference in terms of communication complexity in a stable view since all backup replicas act in the same way, i.e. the protocol still follows all-to-all communication pattern. However, the choice may affect view-change overhead, fault tolerance in a stable view, and slow replicas.
The choice 1 would be optimal for graceful execution since it can proceed with the minimal number of received messages in a stable view. Thus it provides minimal latency and even allows the system to withstand up to n-f-1 faulty replicas in a stable view. However it may incur significantly higher overhead in view change due to the bigger new-view certificate size, since each ViewChange message in the new-view certificate carries the protocol history of the corresponding replica.
The choice 2 would be optimal for view change, due to the minimal new-view certificate size, but the latency in a stable view would be limited by (f+1)-th slowest replica. On the other hand, it would give slow replicas best chances to keep up with the protocol execution.
The choice 3 would represent a simple form of compromise. It may give moderately better latency and fault tolerance in a stable view than the choice 2, whereas moderately reducing view-change overhead. On the other hand, it would still give slow replicas less chance to keep up with the protocol execution than the choice 2.
Would you have a preferred choice or any additional concern?
Beta Was this translation helpful? Give feedback.
All reactions