Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't do mempool lookups for "mempool" command without a filter #7174

Merged
merged 1 commit into from
Dec 5, 2015
Merged

Don't do mempool lookups for "mempool" command without a filter #7174

merged 1 commit into from
Dec 5, 2015

Conversation

TheBlueMatt
Copy link
Contributor

No description provided.

@pstratem
Copy link
Contributor

pstratem commented Dec 5, 2015

utACK

@pstratem
Copy link
Contributor

pstratem commented Dec 5, 2015

This performance regression was introduced in 319b116

@petertodd
Copy link
Contributor

utACK

1 similar comment
@jgarzik
Copy link
Contributor

jgarzik commented Dec 5, 2015

utACK

@gmaxwell
Copy link
Contributor

gmaxwell commented Dec 5, 2015

ACK. Good catch, it makes a huge performance difference with a large mempool. The problem still exists with a filter set, but it's a nice step forward. This change also makes the code simpler and clearer, which is always a bonus. I think we should take this back to 0.12 as well.

@gmaxwell gmaxwell merged commit 96918a2 into bitcoin:master Dec 5, 2015
gmaxwell added a commit that referenced this pull request Dec 5, 2015
@gmaxwell
Merge pull request #7174
075faae 
96918a2 Don't do mempool lookups for "mempool" command without a filter (Matt Corallo)
@gmaxwell gmaxwell added this to the 0.12.0 milestone Dec 6, 2015
@dcousens
Copy link
Contributor

dcousens commented Dec 7, 2015

utACK

@laanwj
Copy link
Member

laanwj commented Dec 7, 2015

Posthumous ACK on the code change.
But this should have had a description and motivation in the commit message and PR description.

laanwj pushed a commit that referenced this pull request Dec 7, 2015
@TheBlueMatt @laanwj
Don't do mempool lookups for "mempool" command without a filter
82aff88 
Github-Pull: #7174
Rebased-From: 96918a2
@luke-jr
Copy link
Member

luke-jr commented Dec 7, 2015

Agreed, I was confused as to its behaviour for a while. The description given sounds like "mempool" wouldn't work at all with non-filter connections.

@jgarzik
Copy link
Contributor

jgarzik commented Dec 7, 2015

Da. The description confused several devs...

@gmaxwell
Copy link
Contributor

gmaxwell commented Dec 7, 2015

Ah, I didn't see the potential for confusion it all. Though the description was complete and accurate, what "mempool lookup" means and is ambigious I could see it how you might extract that from the description without reading the code.

For short patches I try to read the code before the description, so as to not be biased by what it says it does and fail to see what it actually does. I would have expected with three acks from regulars that someone would have pointed out something like that earlier. Live and learn.

@dcousens
Copy link
Contributor

dcousens commented Dec 8, 2015

Title made sense to me, I can understand how the misconception could exist, but, upon reading the code it was clear that it was just moving the branching logic around such that mempool.lookup(...) is only being done when necessary instead of pre-emptively.

dcousens
dcousens reviewed Dec 8, 2015
View reviewed changes
src/main.cpp
bool fInMemPool = mempool.lookup(hash, tx);
if (!fInMemPool) continue; // another thread removed since queryHashes, maybe...
if (!pfrom->pfilter->IsRelevantAndUpdate(tx)) continue;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is 1 other change that wasn't described here, which is !IsRelevantAndUpdate and vInv.size() === MAX_INV_SZ would previously cause a "inv" message to be pushed.
This does not occur now.

However, given the logic that vInv.size() is only increased if isRelevantAndUpdate was true anyway, means AFAIK this shouldn't be an issue.

I guess, that is a minor performance optimization then 👍 😃

@str4d str4d mentioned this pull request Jul 14, 2017
Draft
@str4d str4d mentioned this pull request Dec 19, 2017
Merged
zkbot added a commit to zcash/zcash that referenced this pull request Apr 5, 2018
@zkbot
Auto merge of #2814 - str4d:2074-node-bloom, r=str4d
f6d09aa 
Add NODE_BLOOM service bit

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6579
  - Zcash equivalent of BIP 111
- bitcoin/bitcoin#6652
  - Docs for BIP 111
- bitcoin/bitcoin#7087
- bitcoin/bitcoin#7174
- bitcoin/bitcoin#8709

Part of #2074. Closes #2738.
@furszy furszy mentioned this pull request Dec 31, 2020
Merged
furszy added a commit to PIVX-Project/PIVX that referenced this pull request Jan 23, 2021
@furszy
Merge #2118: [NET] Inv, mempool and validation improvements
1f010c0 
9b9c616 Fix missing zapwallettxes mode in wallet_hd.py functional test (furszy)
d6d0ad9 [logs] fix zapwallettxes startup logs (John Newbery)
006c503 [wallet] fix zapwallettxes interaction with persistent mempool (John Newbery)
c6d45c6 Adapting and connecting mempool_persist.py functional test to the test runner. (furszy)
4f26a4e Control mempool persistence using a command line parameter. (John Newbery)
5d949de [Qt] Do proper shutdown (Jonas Schnelli)
e60da98 Allow shutdown during LoadMempool, dump only when necessary (Jonas Schnelli)
c0a0e81 Moving TxMempoolInfo tx member to CTransactionRef (furszy)
f0c2255 Add mempool.dat to doc/files.md (furszy)
8e52226 Add DumpMempool and LoadMempool (Pieter Wuille)
44c635d Add AcceptToMemoryPoolWithTime function (Pieter Wuille)
6bbc6a9 Add feedelta to TxMempoolInfo (Pieter Wuille)
9979f3d [mempool] move removed and conflicts transaction ref list to vector. (furszy)
4f672c2 Make removed and conflicted arguments optional to remove (Pieter Wuille)
e51c4b8 Bypass removeRecursive in removeForReorg (Pieter Wuille)
54cf7c0 Get rid of CTxMempool::lookup() entirely (furszy)
35bc2a9 Finished the switch CTransaction storage in mempool to CTransactionRef and introduced the timeLastMempoolReq coming from btc#8080. (furszy)
d10583b An adapted version of btc@b5599147533103efea896a1fc4ff51f2d3ad5808 (furszy)
cb4fc6c An adapted version of btc@ed7068302c7490e8061cb3a558a0f83a465beeea (furszy)
9645775 Split up and optimize transaction and block inv queues (furszy)
68bc68f Don't do mempool lookups for "mempool" command without a filter (furszy)
7624823 mapNextTx: use pointer as key, simplify value (furszy)
191c62e Return mempool queries in dependency order (Pieter Wuille)
23c9f3e Eliminate TX trickle bypass, sort TX invs for privacy and priority. (furszy)
6ebfd17 tiny test fix for mempool_tests (Alex Morcos)
8c0016e Check all ancestor state in CTxMemPool::check() (furszy)
91c6096 Add ancestor feerate index to mempool (Suhas Daftuar)
64e84e2 Add ancestor tracking to mempool  This implements caching of ancestor state to each mempool entry, similar to descendant tracking, but also including caching sigops-with-ancestors (as that metric will be helpful to future code that implements better transaction selection in CreatenewBlock). (furszy)
8325bb4 Fix mempool limiting for PrioritiseTransaction Redo the feerate index to be based on mining score, rather than fee. (furszy)
1fa40ac Remove work limit in UpdateForDescendants()  The work limit served to prevent the descendant walking algorithm from doing too much work by marking the parent transaction as dirty. However to implement ancestor tracking, it's not possible to similarly mark those descendant transactions as dirty without having to calculate them to begin with.  This commit removes the work limit altogether. With appropriate chain limits (-limitdescendantcount) the concern about doing too much work inside this function should be mitigated. (furszy)
ba32375 Rename CTxMemPool::remove -> removeRecursive  remove is no longer called non-recursively, so simplify the logic and eliminate an unnecessary parameter (furszy)
c30fa16 CTxMemPool::removeForBlock now uses RemoveStaged (furszy)

Pull request description:

  Ending up 2020 with a large PR :).

  Included a good number of performance and privacy improvements over the mempool and inv processing/sending areas + added mempool cache dump/load.
  Almost finishing with #1726 work, getting closer to 9725, and getting closer to be able to decouple the message processing thread <-> wallet and the wallet <-> GUI locks dependencies (which will be another long story.. but well, step by step).

  The final goal is clear, a much faster syncing process using pivx-qt (a good speed up for pivxd as well), smoother visual navigation when big wallets are syncing, less thread synchronization issues, among all of the improvements that will be coming with the backports adaptations.

  Adapted the following PRs to our sources:

  bitcoin#7062 —> only eb30666.
  bitcoin#7174 —> complete.
  bitcoin#7562 —> only c5d746a.
  bitcoin#7594 —> complete.
  bitcoin#7840 —> complete.
  bitcoin#7997 —> complete.
  bitcoin#8080 —> complete
  bitcoin#8126 —> except e9b4780 (we don't have the mapRelay) and c2a4724 (we don't have the relay expiration vector).
  bitcoin#8448 —> complete
  bitcoin#8515 —> complete
  bitcoin#9408 —> complete
  bitcoin#9966 —> complete
  bitcoin#10330 —> complete

ACKs for top commit:
  random-zebra:
    ACK 9b9c616
  Fuzzbawls:
    ACK 9b9c616

Tree-SHA512: 186bd09bbb19b55ec0d46dc27291663a0df2d60d8238c661a8c9b8cbf3677b6f8a92fa56bd7346a3cb5293712eeccc5d6542ee3c441d35a4f61e7d12e2ce489a
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.12.0
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@TheBlueMatt @pstratem @petertodd @jgarzik @gmaxwell @dcousens @laanwj @luke-jr