Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a block pruner that only prunes occassionally #415

Merged
merged 1 commit into from
Oct 12, 2016

Conversation

cjepson
Copy link
Contributor

@cjepson cjepson commented Oct 10, 2016

The blockchain pruning functions would run at every newly inserted
block node, causing performance degradation. Now the blockchain will
only prune nodes to return the memory to the garbage collector at
prespecified (5 minute) intervals, resulting in much less overhead
when syncing the blockchain.

@cjepson cjepson force-pushed the cj_101016gcless branch 2 times, most recently from 018a514 to e6bc702 Compare October 10, 2016 20:14
@jrick
Copy link
Member

jrick commented Oct 10, 2016

Haven't done a full review but the code appears correct. Only issue I have is this feature is mandatory (probably required to prevent leaks?) and not configurable at all (one might want to run it more often, or based on the number of nodes in memory rather than on a timer).

@alexlyp
Copy link
Member

alexlyp commented Oct 10, 2016

synced testnet in approx 22min, mainnet in 16 min. running smooth otherwise, no noticeable increase in mem usage either

@cjepson
Copy link
Contributor Author

cjepson commented Oct 11, 2016

@jrick I agree it's not the most ideal method, but there is a constant to change if you'd like to for the number of minutes to wait before pruning. It can be improved in the future, but for now I'd like it merged because the syncing improvement is so dramatic.

Copy link
Member

@jrick jrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Work can still be done to make this more flexible for callers but for the common case (usage in dcrd) this improves IBD quite a bit.

@cjepson
Copy link
Contributor Author

cjepson commented Oct 12, 2016

Thanks, going in

The blockchain pruning functions would run at every newly inserted
block node, causing performance degradation. Now the blockchain will
only prune nodes to return the memory to the garbage collector at
prespecified (5 minute) intervals, resulting in much less overhead
when syncing the blockchain.
@cjepson cjepson merged commit c796d13 into decred:master Oct 12, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants