Optimizing consensus syncing time

[vncoelho]

Hey @erikzhang,

We are currently discussing (@igormcoelho) about simple adjustment that can help the Blockchain to carve 15s, without changing the structure of the consensus neither modifying the variable that guides the desired block time.

I will open this PR in order to promote a better environment for us to discuss these minor changes.

In fact, we are dealing with something very simple.
The idea is basically to advance the Speaker OnTimeout timer in something around 3-6 seconds, calling "send prepare request" a little before 15 seconds.

While it implies less time for accepting transactions, we are also discussing a simple Average time calculator, which could use the average value taken by the FillContext(); call and predicts the "optimal" time for broadcasting a Prepare Request call.

What do you think about it?

[igormcoelho]

It's a nice idea @vncoelho . If the average time is taken from the last few blocks, it's possible to create a self-adjustable system that will keep achieving 15 seconds on average, even when transaction demands vary.

[erikzhang]

This can be dangerous because shortening the timeout time can lead to frequent view changing that might destroy the entire network.

[vncoelho]

These guys like to play with fire 🗡️ ehauehauehauuhea

@erikzhang, you are completely right. EHauheuahea
We already created more than 100 private nets testing these ideas and....We are still not confident :(

However, we see some light! 🥇

We think that several times the consensus nodes are changing view without necessity. They should mostly change their views when something happens (fraudulent cases). Under delay, I think that the time could be a little extended for avoiding a changing view loop that could mess with the network.

The strategy could be to increase the time for changing the view, at least, in the first iteration (view = 0).
https://github.com/neo-project/neo/pull/268/files#diff-0285c1c12d1d492897a99ffe07f9fed9R34
It could be more than 30 (which is the normal time for blocktime = 15).

But, perhaps, we could also generalize and adjust the ChangingView Timeout formula.
Thus, maybe we could perform this Speaker timeout advance (3-6s) and delay the changing view timeout (2^(v+1) * Blocktime to something like MIN(2^(v+2)*blocktime,35).

On the other hand, we already tested it with blocktimes even set to 5 seconds and the results were not bad.
However, we do not exhaustive test it under the broadcasting of several TX's. But we now have the tools.
We are just waiting for the pending PR acceptances and, then, we start the experiments again.

We have no hurry with this...Let's move step by step.

[vncoelho]

@igormcoelho, yes, maa broda.
We could use a minmax function, such as MIN( MAX(AVG, minTimeToAdvance), maxTimeToAdvance ), for limiting for the feasible syncing times.

For example, with maxTimeToAdvance = 10 precious seconds
In our shared private net, minTimeToAdvance = 3-4 seconds is quite nice.
For the mainnet maybe it should be something like 4-6 seconds

[vncoelho]

Starting to check this fine-tunning for Neo 3.0

[erikzhang]

@vncoelho Now #288 has been merged. We can now handle this PR. First we need to resolve the conflicts.

[vncoelho]

I tried to manage to clear everything, but github suggest me to create a new Pull Request to be more clear.