Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
blockchain: Convert to direct single-step reorgs. #1500
This requires PR #1471.
This modifies the chain reorganization logic to directly perform the reorg one block at a time with rollback in the case of failure, as opposed to the existing memory-based two-step approach, so that it is more optimized for the typical case, better handles large reorgs, gives the ability to implement better caching strategies, and helps provide a path to decouple the chain processing and connection code from the download logic. It also removes the cached stxos from the view since the aforementioned changes make them no longer necessary.
A side effect of these changes is that it is no longer possible to know if a reorg will succeed before actually performing it, so the
Prior to these changes, chain reorganization used a two-step approach such that the first step involved checking all of the blocks along the reorg path in memory and then actually performing the reorg in a second step if those checks succeeded. While that approach does have some benefits in terms of avoiding any intermediate mutation to the current best chain for failed reorgs, and thus not requiring a rollback in that case, it also has some disadvantages such as not scaling well with large reorgs, being more difficult to make use of different caching strategies, and hindering the ability to decouple the connection code from the download logic.
In a certain sense, the approach this replaces assumed that a reorg would fail and took measures to detect that condition prior to performing the reorg, while the new approach assumes the reorg will succeed and rolls back the changes in the very rare case it doesn't. This is an acceptable and safe assumption because the proof-of-work requirements make it exceedingly expensive to create blocks that are valid enough to trigger a reorg yet ultimately end up failing to connect, thus miners are heavily disincentivized from creating such invalid blocks and attackers are also unable to easily create such blocks either. Even in the case of attack, the only result would be nodes performing slightly more database updates than the existing approach.
The following results show the difference between performing the large reorg full block tests before and after these changes:
As can be seen, the new approach takes much less memory and is also a bit faster as well.
This is work towards #1145.