New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune: Support noncontiguous block files #6221

Merged
merged 1 commit into from Jun 11, 2015

Conversation

Projects
None yet
2 participants
@ajweiss
Contributor

ajweiss commented Jun 2, 2015

In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks. Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest. This allows pruning to work
correctly while downloading any blocks needed during the reindex.

Show outdated Hide outdated src/init.cpp
((it->path().filename().string().substr(0,3) == "blk") ||
(it->path().filename().string().substr(0,3) == "rev")))
boost::filesystem::remove(it->path());
int nContigCounter = 0;

This comment has been minimized.

@laanwj

laanwj Jun 3, 2015

Member

So the goal here is to remove all block files, unless they're contiguous and starting from 0? Maybe add a comment.

@laanwj

laanwj Jun 3, 2015

Member

So the goal here is to remove all block files, unless they're contiguous and starting from 0? Maybe add a comment.

Prune: Support noncontiguous block files
In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks.  Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest.  This allows pruning to work
correctly while downloading any blocks needed during the reindex.
@ajweiss

This comment has been minimized.

Show comment
Hide comment
@ajweiss

ajweiss Jun 3, 2015

Contributor

Ok, I've reworked the comments to make things far more explicit...

Contributor

ajweiss commented Jun 3, 2015

Ok, I've reworked the comments to make things far more explicit...

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 4, 2015

Member

Thanks for adding documentation. utACK.

Member

laanwj commented Jun 4, 2015

Thanks for adding documentation. utACK.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 11, 2015

Member

Tested by @BitpopCoin in #6184, confirmed to fix the issue

Member

laanwj commented Jun 11, 2015

Tested by @BitpopCoin in #6184, confirmed to fix the issue

@laanwj laanwj merged commit c257a8c into bitcoin:master Jun 11, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Jun 11, 2015

Merge pull request #6221
c257a8c Prune: Support noncontiguous block files (Adam Weiss)
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 11, 2015

Member

Cherry-picked to 0.11 as 6cb70ca

Member

laanwj commented Jun 11, 2015

Cherry-picked to 0.11 as 6cb70ca

laanwj added a commit that referenced this pull request Jun 11, 2015

Prune: Support noncontiguous block files
In some corner cases, it may be possible for recent blocks to end up in
the same block file as much older blocks.  Previously, the pruning code
would stop looking for files to remove upon first encountering a file
containing a block that cannot be pruned, now it will keep looking for
candidate files until the target is met and all other criteria are
satisfied.

This can result in a noncontiguous set of block files (by number) on
disk, which is fine except for during some reindex corner cases, so
make reindex preparation smarter such that we keep the data we can
actually use and throw away the rest.  This allows pruning to work
correctly while downloading any blocks needed during the reindex.

Rebased-From: c257a8c
Github-Pull: #6221
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment