Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: filteradd message: update bloom filter empty/full flags after adding #16922

Conversation

theStack
Copy link
Contributor

fixes #16886, discovered by alexzk1
(slight performance improvement)

@DrahtBot DrahtBot added the P2P label Sep 19, 2019
@promag
Copy link
Member

promag commented Sep 22, 2019

ACK 9a2eee9.

The suggestion in #16886 (comment) makes sense but it requires a slightly larger refactor but maybe worth the effort?

@theStack
Copy link
Contributor Author

@promag: Thanks for the review! The suggestion by alexzk1 to make the method private and handle the empty/full flag only internally seemed good to me at first as well, but where would be appropriate places to trigger the update? As you noted, insert() is not an option since it is called in a loop.

@laanwj
Copy link
Member

laanwj commented Sep 30, 2019

(slight performance improvement)

When claiming performance improvements, please provide a benchmark.

@theStack
Copy link
Contributor Author

theStack commented Oct 6, 2019

(slight performance improvement)

When claiming performance improvements, please provide a benchmark.

Okay, will try to craft a benchmark for this within the next days.

…ding

fixes bitcoin#16886, discovered by alexzk1
(slight performance improvement)
@theStack theStack force-pushed the 20190919-net-update_empty_full_after_adding_filter branch from 9a2eee9 to b1dcc89 Compare October 10, 2019 10:25
@theStack
Copy link
Contributor Author

theStack commented Oct 10, 2019

(slight performance improvement)

When claiming performance improvements, please provide a benchmark.

The following benchmarks (theStack@3da6aac -- not part of this PR, for convenience added to rollingbloom.cpp rather than creating a new file bloom.cpp) show how the operations to insert (method .insert()) and check for (method .contains()) elements perform for full bloom filters, once without and once with calling .UpdateEmpyFull() before:

$ src/bench/bench_bitcoin "-filter=BloomFull.*"
# Benchmark, evals, iterations, total, min, max, median
BloomFull_Regular_Contains, 5, 21000000, 11.1822, 1.03667e-07, 1.08973e-07, 1.06582e-07
BloomFull_Regular_Insert, 5, 21000000, 11.3052, 1.05405e-07, 1.1052e-07, 1.06237e-07
BloomFull_UpdateEmptyFull_Contains, 5, 21000000, 0.478308, 3.78305e-09, 5.22676e-09, 4.73157e-09
BloomFull_UpdateEmptyFull_Insert, 5, 21000000, 0.386489, 3.41868e-09, 3.93211e-09, 3.71036e-09

My wording was sloppy -- the performance improvement (or in this case, CPU waste minimizing) is actually not 'slight', but significant (20-30x), but it only applies for full filters, which is not a regular case.

Note that the optimization itself is not new, it was added by gmaxwell with commit 37c6389 ("Performance optimization for bloom filters."). The point is that it was only called on receiving the filterload message, but not after receiving the filteradd message, which is fixed with this PR.

@fanquake
Copy link
Member

fanquake commented Jan 5, 2020

@gmaxwell Would you like to comment here?

@kallewoof
Copy link
Member

I'm confused why I was asked to review this as I haven't touched this code.

@theStack
Copy link
Contributor Author

theStack commented Feb 5, 2020

I'm confused why I was asked to review this as I haven't touched this code.

For some reason you were in the list of suggested reviewers (don't know on what criteria github decides this). Feel free to ignore.

@theStack
Copy link
Contributor Author

At the time of opening this PR I assumed that the reason for the introduction of the CBloomFilter empty/full flag was optimization, as stated in the commit message, but in fact it was a covert fix for CVE-2013-5700 (see gmaxwells comment about commit 37c6389: #18483 (comment)). Hence this PR is closed now.

@theStack theStack closed this Apr 15, 2020
@theStack theStack deleted the 20190919-net-update_empty_full_after_adding_filter branch April 15, 2020 18:37
fanquake added a commit that referenced this pull request May 6, 2020
…rify CVE fix

1ad8ea2 net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix (Sebastian Falbesoner)

Pull request description:

  The BIP37 bloom filter class `CBloomFilter` contains two flags `isEmpty`/`isFull` together with an update method with the purpose to, according to the comments, "avoid wasting cpu", i.e. the mechanism should serve as an optimization for the trivial cases of empty (all bits zero) or full (all bits one) filters.
  However, the real reason of adding those flags (introduced with commit 37c6389 by gmaxwell) was a _covert fix_ of [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700), a vulnerability that allowed a divide-by-zero remote node crash.
  According to gmaxwell himself (#9060 (comment)):
  > the IsEmpty/IsFull optimizations were largely a pretextual optimization intended to make unexploitable a remote crash vulnerability (integer division by zero) that existed in the original bloom filtering code without disclosing it. I'm doubtful that they are all that useful. :)

  For more information on how to trigger this crash, see PR #18515 which contains a detailled description and a regression test. It has also been discussed on a [recent PR club meeting on fuzzing](https://bitcoincore.reviews/18521.html).

  The covert fix code already led to issues and PR based on the wrong assumption that the flags are there for optimization reasons (see #16886 and #16922). This PR gets rid of the flags and the update method and just focuses on the CVE fix itself, i.e. it can be seen as a revert of the covert fix commit modulo the actual fix.

ACKs for top commit:
  meshcollider:
    utACK 1ad8ea2
  laanwj:
    Concept and code review ACK 1ad8ea2
  jkczyz:
    ACK 1ad8ea2
  MarcoFalke:
    ACK 1ad8ea2
  fjahr:
    Code review ACK 1ad8ea2

Tree-SHA512: 29f7ff9faece0285e11e16c024851f5bcb772dec64118ccc3f9067ec256267ec8e1b1e3105c7de2a72fd122c3b085e8fc840ab8f4e49813f1cc7a444df1867f7
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request May 12, 2020
…er, clarify CVE fix

1ad8ea2 net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix (Sebastian Falbesoner)

Pull request description:

  The BIP37 bloom filter class `CBloomFilter` contains two flags `isEmpty`/`isFull` together with an update method with the purpose to, according to the comments, "avoid wasting cpu", i.e. the mechanism should serve as an optimization for the trivial cases of empty (all bits zero) or full (all bits one) filters.
  However, the real reason of adding those flags (introduced with commit bitcoin@37c6389 by gmaxwell) was a _covert fix_ of [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700), a vulnerability that allowed a divide-by-zero remote node crash.
  According to gmaxwell himself (bitcoin#9060 (comment)):
  > the IsEmpty/IsFull optimizations were largely a pretextual optimization intended to make unexploitable a remote crash vulnerability (integer division by zero) that existed in the original bloom filtering code without disclosing it. I'm doubtful that they are all that useful. :)

  For more information on how to trigger this crash, see PR bitcoin#18515 which contains a detailled description and a regression test. It has also been discussed on a [recent PR club meeting on fuzzing](https://bitcoincore.reviews/18521.html).

  The covert fix code already led to issues and PR based on the wrong assumption that the flags are there for optimization reasons (see bitcoin#16886 and bitcoin#16922). This PR gets rid of the flags and the update method and just focuses on the CVE fix itself, i.e. it can be seen as a revert of the covert fix commit modulo the actual fix.

ACKs for top commit:
  meshcollider:
    utACK 1ad8ea2
  laanwj:
    Concept and code review ACK 1ad8ea2
  jkczyz:
    ACK 1ad8ea2
  MarcoFalke:
    ACK 1ad8ea2
  fjahr:
    Code review ACK 1ad8ea2

Tree-SHA512: 29f7ff9faece0285e11e16c024851f5bcb772dec64118ccc3f9067ec256267ec8e1b1e3105c7de2a72fd122c3b085e8fc840ab8f4e49813f1cc7a444df1867f7
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CBloomFilter: Missing updateemptyfull?
6 participants