Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune: Support noncontiguous block files #6221

Merged
merged 1 commit into from Jun 11, 2015

Conversation

ajweiss
Copy link
Contributor

@ajweiss ajweiss commented Jun 2, 2015

In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks. Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest. This allows pruning to work
correctly while downloading any blocks needed during the reindex.

((it->path().filename().string().substr(0,3) == "blk") ||
(it->path().filename().string().substr(0,3) == "rev")))
boost::filesystem::remove(it->path());
int nContigCounter = 0;
Copy link
Member

@laanwj laanwj Jun 3, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the goal here is to remove all block files, unless they're contiguous and starting from 0? Maybe add a comment.

In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks.  Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest.  This allows pruning to work
correctly while downloading any blocks needed during the reindex.
@ajweiss ajweiss force-pushed the ajw_1506_prune_noncontig_blockfiles branch from 588bbb0 to c257a8c Compare Jun 3, 2015
@ajweiss
Copy link
Contributor Author

ajweiss commented Jun 3, 2015

Ok, I've reworked the comments to make things far more explicit...

@laanwj
Copy link
Member

laanwj commented Jun 4, 2015

Thanks for adding documentation. utACK.

@laanwj
Copy link
Member

laanwj commented Jun 11, 2015

Tested by @BitPopCoin in #6184, confirmed to fix the issue

@laanwj laanwj merged commit c257a8c into bitcoin:master Jun 11, 2015
laanwj added a commit that referenced this pull request Jun 11, 2015
c257a8c Prune: Support noncontiguous block files (Adam Weiss)
@laanwj
Copy link
Member

laanwj commented Jun 11, 2015

Cherry-picked to 0.11 as 6cb70ca

laanwj pushed a commit that referenced this pull request Jun 11, 2015
In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks.  Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest.  This allows pruning to work
correctly while downloading any blocks needed during the reindex.

Rebased-From: c257a8c
Github-Pull: #6221
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants