Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make sigcache faster, more efficient, larger #6918

Merged
merged 3 commits into from
Nov 12, 2015
Merged

Make sigcache faster, more efficient, larger #6918

merged 3 commits into from
Nov 12, 2015

Conversation

sipa
Copy link
Member

@sipa sipa commented Oct 30, 2015

  • Changes the -maxsigcachesize argument from entries to megabytes
  • Change the default to 40 MiB (as opposed to ~10 MiB before), corresponding to over 500000 entries on 64-bit systems.
  • Store salted SHA256 hashes in the cache, rather than full entries.
  • Switch implementation from std::set to boost::unordered_set (smaller and faster)
  • Remove cache entries after having been validated inside a block, to keep the cache fresh.

@morcos
Copy link
Member

morcos commented Oct 30, 2015

Concept ACK

Sent from my iPhone

On Oct 30, 2015, at 6:20 PM, Pieter Wuille notifications@github.com wrote:

Changes the -maxsigcachesize argument from entries to megabytes
Change the default to 40 MiB (as opposed to ~10 MiB before), corresponding to over 500000 entries on 64-bit systems.
Store salted SHA256 hashes in the cache, rather than full entries.
Switch implementation from std::set to boost::unordered_set (smaller and faster)
You can view, comment on, or merge this pull request online at:

#6918

Commit Summary

Make sigcache faster and more efficient
File Changes

M src/init.cpp (3)
M src/script/sigcache.cpp (80)
M src/script/sigcache.h (4)
Patch Links:

https://github.com/bitcoin/bitcoin/pull/6918.patch
https://github.com/bitcoin/bitcoin/pull/6918.diff
—
Reply to this email directly or view it on GitHub.

@pstratem
Copy link
Contributor

concept ACK

dcousens
dcousens reviewed Oct 30, 2015
View reviewed changes
src/script/sigcache.cpp

boost::unique_lock<boost::shared_mutex> lock(cs_sigcache);

while (static_cast<int64_t>(setValid.size()) > nMaxCacheSize)
while (memusage::DynamicUsage(setValid) > nMaxSize)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OOI, what is the time complexity of memusage::DynamicUsage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

O(1).

@dcousens
Copy link
Contributor

concept ACK, utACK 👍

dcousens
dcousens reviewed Oct 30, 2015
View reviewed changes
src/script/sigcache.cpp
bool
Get(const uint256 &hash, const std::vector<unsigned char>& vchSig, const CPubKey& pubKey)
Get(const uint256& entry)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get is such a weird name, its really more like Contains?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's going for a getter/setter paradigm(see: Set function below), but I think the names could indeed be improved. It's confusing to have "Set" be both the type of data structure and the primary way of updating it, imo.

@sipa
Copy link
Member Author

sipa commented Oct 31, 2015

Added a commit that removes cache entries after they've been seen in a block, so the cache remains fresh.

@dcousens
Copy link
Contributor

re-ACK

Diapolo
Diapolo reviewed Oct 31, 2015
View reviewed changes
src/script/sigcache.cpp
class CSignatureCacheHasher
{
public:
size_t operator()(const uint256& key) const {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a qustion to understand the motivation, why is here the opening bracket in the same line and below (line 48) it's on a new line?

@sipa
Copy link
Member Author

sipa commented Nov 2, 2015

Added a commit to not wipe the cache from TBV.

@sipa
Copy link
Member Author

sipa commented Nov 2, 2015

Currently gathering statistics about the cache size - hitrate relation.

@instagibbs
Copy link
Member

utACK

@gmaxwell gmaxwell added this to the 0.12.0 milestone Nov 5, 2015
@sdaftuar
Copy link
Member

sdaftuar commented Nov 6, 2015

Concept ACK. However in my first attempt at benchmarking I'm not seeing any speedup with this code change (in fact block verification times seem to have gotten very, very slightly slower for some reason). Will continue to debug and try to see if my benchmarking is broken, or if there's some effect here we're missing.

@morcos morcos mentioned this pull request Nov 9, 2015
Closed
@sipa
Copy link
Member Author

sipa commented Nov 11, 2015

@sdaftuar The results #6976 indicate that this improves things. Does that contradict your earlier findings?

@sdaftuar
Copy link
Member

Sorry forgot to update with my results. I ran longer studies comparing the old and new behavior more carefully and yes I agree this does improve things. I saw roughly a 5% speed up in processing all the October historical data using the new default cache size compared with the old implementation (running with a sig cache of approximately the same size).

ACK

@gmaxwell
Copy link
Contributor

ACK

@laanwj laanwj merged commit 69d373f into bitcoin:master Nov 12, 2015
laanwj added a commit that referenced this pull request Nov 12, 2015
@laanwj
Merge pull request #6918
eb6172a 
69d373f Don't wipe the sigcache in TestBlockValidity (Pieter Wuille)
0b9e9dc Evict sigcache entries that are seen in a block (Pieter Wuille)
830e3f3 Make sigcache faster and more efficient (Pieter Wuille)
@dagurval dagurval mentioned this pull request Mar 19, 2018
Merged
@str4d str4d mentioned this pull request May 14, 2018
Merged
zkbot added a commit to zcash/zcash that referenced this pull request May 14, 2018
@zkbot
Auto merge of #3262 - str4d:2074-perf-1, r=<try>
8ce4bc1 
Bitcoin 0.12 performance improvements

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6918
- bitcoin/bitcoin#6932

Part of #2074.
zkbot added a commit to zcash/zcash that referenced this pull request May 31, 2018
@zkbot
Auto merge of #3262 - str4d:2074-perf-1, r=<try>
03532b1 
Bitcoin 0.12 performance improvements

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6918
- bitcoin/bitcoin#6932

Part of #2074.
zkbot added a commit to zcash/zcash that referenced this pull request Nov 30, 2018
@zkbot
Auto merge of #3262 - str4d:2074-perf-1, r=str4d
9cd7486 
Bitcoin 0.12 performance improvements

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6918
- bitcoin/bitcoin#6932

Part of #2074.
milesmanley pushed a commit to RunOnFlux/fluxd that referenced this pull request Mar 5, 2019
@zkbot @milesmanley
Auto merge of #3262 - str4d:2074-perf-1, r=str4d
67d2fb1 
Bitcoin 0.12 performance improvements

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6918
- bitcoin/bitcoin#6932

Part of #2074.
@furszy furszy mentioned this pull request Jun 8, 2020
Merged
furszy added a commit to PIVX-Project/PIVX that referenced this pull request Jun 18, 2020
@furszy
Merge #1668: [Backport] Make sigcache faster, more efficient, larger
7e65a82 
6706c20 Don't wipe the sigcache in TestBlockValidity (furszy)
c69e354 Evict sigcache entries that are seen in a block (Pieter Wuille)
fb3a4a6 Make sigcache faster and more efficient (Pieter Wuille)

Pull request description:

  Coming from bitcoin#6918 .

  > Changes the -maxsigcachesize argument from entries to megabytes
  > Change the default to 40 MiB (as opposed to ~10 MiB before), corresponding to over 500000 entries on 64-bit systems.
  > Store salted SHA256 hashes in the cache, rather than full entries.
  > Switch implementation from std::set to boost::unordered_set (smaller and faster)
  > Remove cache entries after having been validated inside a block, to keep the cache fresh.

  -----

  This is the first step towards an up-to-date sigcache. Next one will be to back port the CuckooCache implementation.

ACKs for top commit:
  random-zebra:
    ACK 6706c20
  Fuzzbawls:
    utACK 6706c20

Tree-SHA512: 3b96c6a9bbbfc5c335644bf66a3605575f66ace88046411386c6a28eee37a6537bac7e25e3eebc4a2b07a82b918385ac82b68349655ab3d7e32a018ef240a908
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
Validation
Projects
None yet
Milestone
0.12.0
Development

Successfully merging this pull request may close these issues.
9 participants
@sipa @morcos @pstratem @dcousens @instagibbs @sdaftuar @gmaxwell @Diapolo @laanwj