Check Steem issue 2658: Not producing block because node didn't wake up within 500ms of the slot time

[abitmore]

steemit/steem#2658

Brief:

• witness_node wakes up on every second to check whether it's time to produce a block; when block interval is 3 seconds, theoretically the node will try 3 times to produce its block on its time slot;
• if the node missed the first chance (the first second), E.G. when latency of previous block is too high, on the 2nd and 3rd second, it's guaranteed (by the code) to fail due to "didn't wake up within 500ms".

bitshares-core/libraries/plugins/witness/witness.cpp

Lines 269 to 273 in af118f6

	if( llabs((scheduled_time - now).count()) > fc::milliseconds( 500 ).count() )
	{
	capture("scheduled_time", scheduled_time)("now", now);
	return block_production_condition::lag;
	}

Generally, if latency of previous block is too high, the node will try to produce another block on the first second of its time slot. But in the scenario when witness list got shuffled after the previous block, the node may "suddenly" find it's its turn to produce next block, but will fail as described above.

Note: this is a minor issue, IMHO low priority.

Possible solution:

• loose the "500ms" limitation, for example, to "block interval" or so?
• Or just remove the check?

Please discuss.

[pmconrad]

We should define desirable behaviour before implementing fixes.

IMO, if a node hasn't received a block from its predecessor when its time slot has come, it should simply produce its own block in time.

• Waiting until its own slot is nearly over is counterproductive, this will only propagate the problem to the next witness in line.
• Producing a block on time is also desirable from the user's perspective, because that will get his transaction approved more quickly.
• Not waiting for the previous block increases the chance of a fork between the previous block and the next. Waiting for the previous block increases the chance of a fork between this block and the next. So in the end waiting doesn't gain us anything (but increases the risk if the cause is not latency but witness failure).

There is one special case though: if the previous block has been received in time but takes a long time to apply (as is often the case in a maintenance block), then it makes sense to produce one even if our slot is nearing its end. I would extend the deadline in that special case only.

[abitmore]

@pmconrad I think your comment is not about the issue in OP, but more related to #504.

In this issue, the node was not waiting doing nothing until it's slot is nearly over, but on the contrary, it tried to produce a block on the 2nd second and the 3rd second but failed due to the timeout check. The reason why it didn't try to produce on the 1st second, is it was not its time slot to produce before received the high-latency block, that said, the high-latency block caused a schedule change.

[pmconrad]

Yes, I thought of a different scenario. Makes sense for a schedule change as well.

[abitmore]

Fixed by #1266.