DAWN-397 ⁃ Most Hated Block (MHB) Fix

[blockone-syncclient]

Fix for the Most Hated Block (MHB) issues encountered over the holiday break. In block 936299.

Some historical context. On Dec. 26, 2017, a message involving the transfer of currency value between two accounts was submitted. This was not a unique event, nor was it ill formed. Yet the WASM runtime infrastructure is unable to process this message, raising a "integer divide by 0 or integer overflow" exception. The problem is that when this first occurred, 15 of the block producers accepted the original block and made it irreversible. However the remaining 6 producer nodes, 4 11 12 14 15 21, all failed to accept the block. Subsequent efforts to restart those nodes also failed, hence the new designation of "Most Hated Block." Eventually I got the testnet restarted by performing a "brain transplant" copying the blockchain database and log files from one of the working nodes to the failing nodes.

The proposed patch provides a workaround which is to catch the specific exception when thrown by the WASM runtime while not producing new blocks. Then just log that the exception occurred but otherwise don't do anything about it.

ATP: start a eosd instance and connect to testnet 1.
Observe that your node syncs past block #936299, the most hated block.
Break into your happy dance. :-)

[blockone-syncclient]

➤ Phil Mesnier commented:

Code for review is in github pull request #1034