Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockchain: Optimize exported header access. #1273

Merged
merged 1 commit into from
Jun 12, 2018

Conversation

davecgh
Copy link
Member

@davecgh davecgh commented Jun 9, 2018

This optimizes the FetchHeader function to make use of the fact that all block nodes are now in memory and therefore it is no longer necessary to consult the database for them.

It also renames the function to HeaderByHash so it is consistent with other similar functions such as HeaderByHeight, BlockByHash, and BlockByHeight and updates all callers in the repo accordingly.

This speeds up fetching all of the headers in the chain by roughly 3x to 4x for a chain height of 246,000 headers. Longer chains will benefit more.

For example, here is some timing information before and after this commit plus the full block index in memory commit for 246,000 headers:

7200 RPM HDD:

Previous fetch time: ~15.76s
New fetch time: ~4.17s

SSD:

Previous fetch time: ~12.63s
New fetch time: ~4.17s

This is work towards #1145.

@davecgh davecgh mentioned this pull request Jun 9, 2018
33 tasks
Copy link
Member

@dajohi dajohi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

This optimizes the FetchHeader function to make use of the fact that all
block nodes are now in memory and therefore it is no longer necessary to
consult the database for them.

It also renames the function to HeaderByHash so it is consistent with
other similar functions such as HeaderByHeight, BlockByHash, and
BlockByHeight and updates all callers in the repo accordingly.

This speeds up fetching all of the headers in the chain by roughly 3x to
4x for a chain height of 246,000 headers.  Longer chains will benefit
more.

For example, here is some timing information before and after this
commit plus the full block index in memory commit for 246,000 headers:

7200 RPM HDD:
-------------
Previous fetch time: ~15.76s
New fetch time: ~4.17s

SSD:
----
Previous fetch time: ~12.63s
New fetch time: ~4.17s
@davecgh davecgh merged commit a235b83 into decred:master Jun 12, 2018
@davecgh davecgh deleted the blockchain_header_by_hash branch June 12, 2018 21:03
@davecgh davecgh added this to the 1.3.0 milestone Jun 12, 2018
cfromknecht added a commit to cfromknecht/btcd that referenced this pull request Aug 1, 2018
backport of decred/dcrd#1273

Notable difference being that btcd mainline currenlty
doesn't have a blockchain/blockindex_test file, so
those changes are ommitted.

Great work @davecgh :)
cfromknecht added a commit to cfromknecht/btcd that referenced this pull request Aug 1, 2018
backport of decred/dcrd#1273

Notable difference being that btcd mainline currenlty
doesn't have a blockchain/blockindex_test file, so
those changes are ommitted.

Great work @davecgh :)
cfromknecht added a commit to cfromknecht/btcd that referenced this pull request Aug 1, 2018
backport of decred/dcrd#1273

Notable difference being that btcd mainline currenlty
doesn't have a blockchain/blockindex_test.go file, so
those changes are omitted.

Great work @davecgh :)
cfromknecht added a commit to cfromknecht/btcd that referenced this pull request Aug 1, 2018
backport of decred/dcrd#1273

Notable difference being that btcd mainline currenlty
doesn't have a blockchain/blockindex_test.go file, so
those changes are omitted.

Great work @davecgh :)
anchaj pushed a commit to phoreproject/btcd that referenced this pull request Sep 24, 2018
backport of decred/dcrd#1273

Notable difference being that btcd mainline currenlty
doesn't have a blockchain/blockindex_test.go file, so
those changes are omitted.

Great work @davecgh :)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants