Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GRPH-75] node restore improvments #101

Merged
merged 15 commits into from
Sep 18, 2019
Merged

[GRPH-75] node restore improvments #101

merged 15 commits into from
Sep 18, 2019

Conversation

gladcow
Copy link

@gladcow gladcow commented Sep 2, 2019

Backport of the Bitshares bitshares/bitshares-core#339

Tl;dr: if during startup we have a (possibly old) object_database available, we should use it.

We have up to two on-disk representations of the blockchain state: the block log and the object_database. The object database can be reconstructed from the block log, this is what a "replay" does. A replay takes a long time, and this time is constantly growing. This PR is an attempt to mitigate that.

During startup, the application checks the availability and state of the on-disk representation. The old code applies some rather restrictive logic (object_database.head_block == block_log.last_block) to determine if the state from the object_database can be used. This prevents creating and using snapshots of the object_database.

The logic implemented by this PR is more natural, IMO:

If we have an object_database, we should use it. Only if object_database.head_block > block_log.last_block we have an inconsistency (we cannot reconstruct that object_database from that block_log). In that case, treat the object_database as if it were absent.
If we don't have an object_database, use genesis.
From where we are now (genesis or object_database.head), replay all additional blocks from the block_log.
If during step 3 we come across block_log.last - 10000 (an arbitrary number about 8 hours in the past), flush the object_database to disk, and continue. This snapshot will be available in the next restart, if the node crashes or does otherwise not exit cleanly.
During step 3, at 50 blocks before block_log.last (another arbitrary number, about twice the number of active witnesses), switch from apply()ing blocks to push()ing blocks. This will populate the fork_database and enable switching to a different fork, if our block_log ends in a fork.

This change will greatly improve startup time for nodes, which will be particularly important ifwhen we experience another blockchain halt, like a couple of weeks ago.

@gladcow gladcow changed the title [GRPH-75] node restore improviments [GRPH-75] node restore improvments Sep 2, 2019
Copy link

@oxarbitrage oxarbitrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good. i tested it and seems to be working as expected.

Peerplays don't have the --plugin option for its witness_node. To remove bookie one actually have to remove the line where the plugin is enabled in programs/witness/main.cpp. This is inconvenient and i am pretty sure this option will be added. This is not in the scope of this pull request.

Good work.

@oxarbitrage oxarbitrage merged commit 96161db into develop Sep 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants