Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[WIP] Mempool: rework rebroadcast logic to improve privacy #16698
The current rebroadcast logic is terrible for privacy because only the source wallet will rebroadcast transactions, and does so quite frequently. This PR aims to improve privacy dramatically while preserving the benefits of rebroadcasting that ensure txns are successfully propagated through the network.
This PR introduces changes so nodes will resend transactions that it believes should have already been mined. It extracts the logic from the wallet into the mempool, so nodes will rebroadcast txns regardless of the originating wallet. Txns are defined as "should have been mined" by using the block assembler logic, and excluding txns that were recently added to the mempool. The wallet will resubmit txns to the mempool on a regular cadence to ensure those txns aren't dropped (due to eviction, expiry, etc..) before being confirmed.
For more information, see: https://gist.github.com/amitiuttarwar/b592ee410e1f02ac0d44fcbed4621dba
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
This code is ready for initial review.
there are still some to dos before it would be ready for merge:
there are also some follow-ups that can be addressed in separate PRs:
Aug 23, 2019
TX fees and policy
Aug 24, 2019
mzumsande left a comment •
I don’t understand 1) the concept “should have been mined”, and 2) how the block assembler logic achieves this.
thanks for the review @mzumsande !
based on the local mempool, we are attempting to answer the question of what txns we think should have been mined. And saying if it wasn't, maybe there was an issue with relay.
yes. you will start with txns you expect to be mined in the next block. the recency filter will (likely) remove some of those transactions. however, in the case of a mempool thats emptying out, the recency filter might not have much effect. for that I have this todo before the PR would be ready for merge:
What confuses me is how we can answer that without actually looking into recent blocks.
How could we distinguish between these cases by just looking at our current mempool?
great questions @mzumsande. I've thought about this a lot, so let me share some of my reasoning-
We can't. And further, even if we do look at the recent blocks, we still cannot answer exactly what "should" have been included. The two main pieces of information we are missing are 1. what the miner's mempool looked like and 2. any weight applied through
yup, specifically in the case of the emptying out mempool, we would currently rebroadcast a full set of txns, thats why I want to build a cache of min fee rate and apply an extra filter to reduce noise in this circumstance ( from #16698 (comment) ):
The crux of the thinking is that we are not going to get the rebroadcast set perfect, but that is ok. When a node rebroadcasts a txn, it sends an INV message to new connections (see
All these different mechanisms are to reduce noise. I want to ensure the parameters chosen allow the worst case scenario (rebroadcasting the full set) to not be disruptive to the network. And these mechanisms should make the worst case infrequent.
Does this make sense to you? Let me know if you still have questions.
Thanks for the answer! I think that your approach makes sense and am looking forward to the traffic/bandwidth analysis.