New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to a more efficient rolling Bloom filter #7113

Merged
merged 1 commit into from Dec 3, 2015

Conversation

Projects
None yet
3 participants
@sipa
Member

sipa commented Nov 27, 2015

For each 'bit' in the filter we really maintain 2 bits, which store either:

  • 0: not set
  • 1-3: set in generation N

After (nElements / 2) insertions, we switch to a new generation, and wipe entries which already had the new generation number, effectively switching from the last 1.5 * nElements set to the last 1.0 * nElements set.

This is 25% more space efficient than the previous implementation, and can (at peak) store 1.5 times the requested amount of history (though only 1.0 times the requested history is guaranteed).

The existing unit tests should be sufficient.

@laanwj laanwj added the Feature label Nov 27, 2015

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Nov 27, 2015

Member

Nice!
(discussion was in Replace setInventoryKnown with a rolling bloom filter. #7100 )

Member

laanwj commented Nov 27, 2015

Nice!
(discussion was in Replace setInventoryKnown with a rolling bloom filter. #7100 )

Switch to a more efficient rolling Bloom filter
For each 'bit' in the filter we really maintain 2 bits, which store either:
0: not set
1-3: set in generation N

After (nElements / 2) insertions, we switch to a new generation, and wipe
entries which already had the new generation number, effectively switching
from the last 1.5 * nElements set to the last 1.0 * nElements set.

This is 25% more space efficient than the previous implementation, and can
(at peak) store 1.5 times the requested amount of history (though only
1.0 times the requested history is guaranteed).

The existing unit tests should be sufficient.
@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Nov 29, 2015

Contributor

ACK

Contributor

pstratem commented Nov 29, 2015

ACK

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost commented Nov 30, 2015

Yes

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Dec 3, 2015

Member

utACK

Member

laanwj commented Dec 3, 2015

utACK

@laanwj laanwj merged commit 086ee67 into bitcoin:master Dec 3, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Dec 3, 2015

Merge pull request #7113
086ee67 Switch to a more efficient rolling Bloom filter (Pieter Wuille)

@dgenr8 dgenr8 referenced this pull request Dec 29, 2015

Merged

Further work on thin blocks #109

codablock added a commit to codablock/dash that referenced this pull request Sep 16, 2017

Merge pull request #7113
086ee67 Switch to a more efficient rolling Bloom filter (Pieter Wuille)

codablock added a commit to codablock/dash that referenced this pull request Sep 19, 2017

Merge pull request #7113
086ee67 Switch to a more efficient rolling Bloom filter (Pieter Wuille)

codablock added a commit to codablock/dash that referenced this pull request Dec 9, 2017

Merge pull request #7113
086ee67 Switch to a more efficient rolling Bloom filter (Pieter Wuille)

codablock added a commit to codablock/dash that referenced this pull request Dec 9, 2017

Merge pull request #7113
086ee67 Switch to a more efficient rolling Bloom filter (Pieter Wuille)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment