Deal with block reorgs

[ltardivo]

TODO at DB level

• Add block_hash to output and input table
• Create blocks_header table (block_height, block_hash, prev_hash?)
• Add new table syncBlockStatus (block_height)
• Rename table syncStatus to syncTxStatus

PROCESSES

SYNC_TX (parallel): KEEP SYNCING

• But instead of asking to RPC the next block, get it from DB blocks_header table
• Always insert into "Input" and "Output" tables. If "duplicated key" error is received over output.pk, the zmq already inserted the rows. This means we should update the block_hash value of all the rows of that block_id.

SYNC_BLOCK (sequential):

• Fill block_header table UNTIL REACHING THE TIP (do not confuse with the initialization process for blockheads download, both cases we receive an error from the node)

• Detect block reorganization (for each block synced):

• Get from RPC the current_block to sync

• Select previous block and get its hash: #"SELECT block_hash FROM block_headers where block_height == current_block -1"

• Check if the current_block.prev_hash matches the previous block hash. If not, launch "searchChainForkOrigin(current_block)" function

• Search the origin of the fork (current_block)

• Ask to RPC for the current_block. Compare with block_headers, if different, continue backwards (get current_block -1 from RPC, compare with block_headers. If different, continue backwards...and so on). If matching, mark range of blocks [found + 1, current_block] to be deleted, and resync from block "found".

• In resume status, check if there are orphaned blocks, update input and output blockHashes, delete blocks

AGED TXs on mempool

• Periodically check all tx over mempool to remove the aged ones (see RPC getrawmempool)

Some example explanied on block reogrs
https://medium.com/@nicolas.vargas_35315/what-are-blockchain-reorgs-and-how-to-detect-them-when-consuming-nodes-762300f4512e

RPC getchaintips
https://developer.bitcoin.org/reference/rpc/getchaintips.html

How to simulate a block reorg on regtest
https://bitcoin.stackexchange.com/questions/114044/how-can-i-simulate-a-reorg-for-testing