Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fastsync with bonsai #2372

Merged
merged 7 commits into from
Jun 22, 2021
Merged

Conversation

matkt
Copy link
Contributor

@matkt matkt commented Jun 3, 2021

Signed-off-by: Karim TAAM karim.t2am@gmail.com

PR description

If we stop and restart a fastsync synchronization we may have missing data. The idea of this PR is to reset the worldstate while keeping the segments to avoid any data corruption

Fixed Issue(s)

Changelog

matkt added 4 commits June 3, 2021 12:34
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
@matkt matkt marked this pull request as ready for review June 14, 2021 13:50
@matkt matkt changed the title test fix fastsync bonsai Fix fastsync with bonsai Jun 14, 2021
Copy link
Contributor

@garyschulte garyschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, we take a big hit for restarting a bonsai fast-sync. Maybe we can revisit this in the future to see if we can do some checkpointing, but this seems like a safe workaround.

@@ -79,7 +80,9 @@
"Fast sync was requested, but cannot be enabled because the local blockchain is not empty.");
return Optional.empty();
}

if (worldStateStorage instanceof BonsaiPersistedWorldState) {
worldStateStorage.clear();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we bail and restart the world state sync if there was an aborted fast sync, but resume the chain downloading? The worldstate is the bulk of the download activity right? the pivot block isn't going to be too far back in history?

If I understand correctly - this seems safe but an expensive restart.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it is a workaround waiting to find better. There are two things we download the blocks headers and the recent state trie. The only part we reset is the recent state trie. The headers are not removed and it is a big part so we reduce the impact to the minimum. And the pivot block remains the same .

This is a fix to avoid that a person who restarts his node during a fastsync does not waste time synchronizing a corrupted node. But the typical use case is not to re-start a node during a fastsync 🧑‍🏭

@matkt matkt enabled auto-merge (squash) June 22, 2021 08:17
@matkt matkt merged commit 012eff0 into hyperledger:master Jun 22, 2021
jflo pushed a commit to jflo/besu that referenced this pull request Jun 25, 2021
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
@matkt matkt deleted the feature/fix-bonsai-fast-sync branch July 13, 2021 12:19
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants