Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance optimization for bloom filters. #2914

Merged
merged 1 commit into from Aug 20, 2013

Conversation

@gmaxwell
Copy link
Contributor

commented Aug 20, 2013

This reduces a peer's ability to attack network resources by
using a full bloom filter, but without reducing the usability
of bloom filters. It sets a default match everything filter
for peers and it generalizes a prior optimization to
cover more cases.

@gmaxwell

This comment has been minimized.

Copy link
Contributor Author

commented Aug 20, 2013

Tested multibit and full node sync as well as a number of hours of regular network operation. More testing with spv nodes would be helpful.

@mikehearn

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2013

Code looks good to me. I haven't tested it in practice yet.

@sipa

This comment has been minimized.

Copy link
Member

commented Aug 20, 2013

Looks good in general to me, but go fix the tests :)

This reduces a peer's ability to attack network resources by
 using a full bloom filter, but without reducing the usability
 of bloom filters.  It sets a default match everything filter
 for peers and it generalizes a prior optimization to
 cover more cases.
@gmaxwell

This comment has been minimized.

Copy link
Contributor Author

commented Aug 20, 2013

@sipa ::nods:: As you wish. :) (where is my brown paper bag?)

@BitcoinPullTester

This comment has been minimized.

Copy link

commented Aug 20, 2013

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/37c6389c5a0ca63ae3573440ecdfe95d28ad8f07 for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

@wtogami

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2013

FWIW, to help with sanity testing, Litecoin deployed this in testing on nodes with hundreds of of peer connections, both with and without bloom enabled.

@gavinandresen

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2013

ACK, ran this overnight on my Mac with no drama.

gavinandresen added a commit that referenced this pull request Aug 20, 2013
Performance optimization for bloom filters.
@gavinandresen gavinandresen merged commit 5250fdc into bitcoin:master Aug 20, 2013
IntegralTeam pushed a commit to IntegralTeam/bitcoin that referenced this pull request Jun 4, 2019
* Store rs_t key time in big endian

Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.

The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.

* Write batch in CleanupOldRecoveredSigs when it gets too large

This avoids RAM filling up and OOM getting triggered.

* Keep track of when a vote was written to the DB and clean up after week

Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.

* Apply suggestions from code review

Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.