add coredev april 2023

bitcoin-core-dev-tech/2023-04-25-mempool-clustering.md

@@ -0,0 +1,216 @@
+---
+title: Mempool Clustering
+categories: ['core-dev-tech']
+tags: ['bitcoin core', 'mempool']
+date: 2023-04-25
+speakers: ['Suhas Daftuar', 'Pieter Wuille']
+---
+
+## Current Problems
+
+lot of problems in the mempool
+
+1. eviction is broken
+2. mining algorithm is part of the problem, it’s not perfect
+3. RBF is like totally broken we complain all the time, sometimes we do/don't RBF when we should/shouldn't
+
+## Eviction
+
+- Eviction is when mempool is full, and we want to throw away the worst tx.
+- Example, we think a tx is worst in mempool but it’s a descendant of a "good" tx.
+- Mempool eviction is kinda the opposite of the mining algorithm.
+- For example the first thing you'd evict may be the first tx you'd mine
+- The first thing you'd evict is the small fee parent, but its descendant fee rate is desirable for mining.
+
+## Mining algorithm has problems
+
+- Mining algorithm is something we can't run in reverse
+- What you wanna do is run mining algorithm on whole mempool - look at top for block, bottom for eviction, but this won’t work because of processing time.
+
+- We don't know what fee rate tx will be included in blocks.
+- The current mining algorithm is quadratic - infeasible for mempool eviction
+
+## RBF
+
+- Compare incoming fee rate tx with conflicting tx
+- We do NOT look at the child tx fee rates
+  - PR [#26451](https://github.com/bitcoin/bitcoin/pull/26451) starts to fix this
+- BIP125 rules are a bit silly (e.g. no new unconfirmed parents)
+- None of the fee rates we talk about in mempool tell us where a tx will end up in a block
+
+"so let's do that"
+
+## TOTAL ORDERING
+
+- These are improvements to block mining will also then improve mempool eviction, etc
+- This should make RBF much much better
+- Eliminate the uncertainty about how good a tx is
+- We can compare 2 txs and decide which is better
+- It won’t totally solve pinning, total fee rate rule is still an issue, v3 relay addresses that
+
+- "obvious" solution - at all points in time, sort mempool
+- ancestor fee rate solution is already n^2, optimal algorithm is clearly exponential, so that wouldn't work to run on whole mempool
+
+- why don't we just limit size of connected components of mempool?
+- Imagine the mempool is a graph, edges between parent/child txs
+- Sort each connected component separately, then do something simple to merge all those together
+
+- Clusters can’t affect each other because they are not connected
+
+- Run quadratic algorithm on every connected component of the mempool
+
+- This introduces a new policy rule
+- Today there is no cluster limit, entire mempool could just be one huge cluster
+
+- “Introducing”: cluster size limit. Is this something users can deal with?
+- It could mean unrelated people have txs related by cluster
+
+- Right now there is max 25 ancestor count
+- We would probably still need an ancestor limit, but maybe not need a descendant limit anymore
+- We might need some kind of carve out like CPFP too
+
+## Definitions
+
+- cluster: set of txs connected in graph
+- linearization: any topologically valid sort of txs in mempool
+
+- normally we talk about it within a cluster
+- block building is linearization as well
+- lots of ways to do this (ancestor feerate-based algorithm is what we use today)
+
+- f(cluster) -> linearization
+- This tells you in what order to include things
+
+## Steps:
+
+1. Identify clusters in mempool
+2. linearize each cluster
+3. ?
+
+- hard part is done, we figured out the best strategy for all the dependencies
+- block template - use that information to pull off the best fee rate parts of each cluster until block is full
+
+## chunks
+
+Given a linearization, we can calculate where the chunk breaks are, where we pick txs into the block. We always start at beginning, but only go in as far as you want to go to maximize fee rate. Look at all the prefixes to pick highest fee rate, everything is safe to cut off, but where is the most optimal cut point… always start on the left.
+
+## Chunk Example
+
+- where could this linearization have come from?
+- Once you have the linearization, the original graph doesn't matter any more
+- The linearization function adds redundant dependencies
+
+## How do we calculate the chunks
+
+Starting at the left, calculate fee rate as you go in to the block (moving to the right) fee rates go up, then down (stop at the peak point). For every prefix, you compute fee rate for each prefix
+then see which prefix has the highest fee rate.
+(It’s n values, not 2^n values because we respect the linearization sort)
+
+1st chunk has been selected, now we look at fee rates for txs after the first chunk.
+“example looks quadratic, but it is actually a linear algorithm”
+
+## Back to Mining
+
+- We get our clusters, we linearize the clusters
+- (what do you do when you violate sigops limit ?!) we need a heuristic to prevent that from happening, i.e. use a modifier to multiply * tx size if we think the tx has more sigops than we think it should have. It’s significantly harder to include sigops because then you're optimizing for two values not just one.
+
+- The problem still gets knapsack-y towards the end, maybe use packages that are small relative to the block
+
+- Anyway, eviction is now the EXACT opposite of the mining algorithm.
+
+- Chunks are usually descending in fee rate, because if they weren't - you would have merged those in to one chunk. Top chunk will be highest fee rate in the cluster. So for eviction, look at the last chunk in each cluster, therefore we evict the last things we would mine.
+
+- The mempool always organizes chunks, mining picks top chunks, eviction picks bottom chunks.
+Within a cluster, chunks are always decreasing fee rate. We can think of the mempool as just one giant list of chunks. We maintainthe data structure in mempool all the time, as new txs come in.
+
+## RBF policy
+
+- We ensure that chunk feerate of new thing is better than chunk fee rate of everything that would be evicted. By talking about chunk fee rate, we are using same score the mining algorithm uses. For a new tx, you look if it has parents and grab all the clusters. A new tx can merge clusters together. Throw new tx into the cluster, sort it, figure out which chunk new tx would be in, then you already have the mining scores of everything about to be evicted.
+
+- So we can create a new mini mempool (just a fake virtual cluster) for each new incoming tx to test its cluster properties.
+
+## "What about RBF carve out?"
+
+- Virtual cluster is affected by cluster size limit as well. Each new incoming tx might merge clusters into a too-big cluster.
+
+- Higher fee rate, higher total fee, we still need these rules for network DoS
+- When you evict something, you need to re-linearize the cluster. We still need some kind of limit so we don't have to re-sort the whole mempool when RBF happens. The clusters don't have an ordering with respect to each other. We only sort the clusters during mining or eviction, but maybe we can optimize the tracking best chunk of each cluster.
+
+- Each tx has its own chunk fee rate based on which chunk it appears in based on the cluster its in. This is its individual score, used for RBF.
+
+## Revisiting Our Problems
+
+- We no longer have asymmetry between mining & eviction
+- If we limit cluster size enough maybe we could run the mining algorithm on the cluster
+
+- "if you want to get your tx mined, don't make a cluster" ?
+- It’s no longer simple to RBF your txs anymore. It doesn't matter though, people follow simple rules: "if you're paying more it'll get picked.” So… linearization algorithms should allow what people typically will do, maybe they CPFP with a few children. After that, it’s just about attack prevention. Can an attacker create a cluster that triggers not-incentive-compatible behavior?
+An attacker can throw your tx into a large cluster.
+
+## Open Questions I
+
+- BIP 125 was rules and terms that users could understand, that produce deterministic results. You could tell ahead of time what will work, but yes you need a mempool to do so. This new approach is more opaque, running an expensive algorithm on the cluster, sorting, etc - turns the algorithm into a black box. We could have an RPC that returns the scores… BIP125 is already too hard to model, and it’s not incentive compatible. The thing that people do: “bumpfee, bumpfee, bumpfee” until it relays.
+
+- "What is the cost of memory usage for this algorithm?"
+  - We don't know, probably negligible.
+
+- Just because we see large clusters doesn't mean anyone really needs them. Its probably OK to split up clusters between blocks. A cluster limit is worse than a descendant limit, but if you risk running in to one, you risk running in to the other.
+
+- Maybe a user wants to fee-bump two separate txs but by bumping them, it joins two clusters together which now violates the limit. (open question)
+
+- Maybe cluster limit is like 50 or 100, replacing descendant limit of 25
+
+## Open Questions II
+
+- sibling limit
+- simple: reject anything that would bust cluster limit
+- We’re punting on sibling eviction for now - could be another pinning attack where attacker can evict your tx by merging clusters, and we still have to manage total relay fee.
+
+"next block fee rate" can be confusing now too, and coin selection gets harder if you're spending unconfirmed coins:
+
+- Wallet need to figure out what cluster each coin is in
+- Choosing coin A might re-score coin B that I might also want to use
+- "That cluster is full so you can’t spend from it at all", etc
+
+3rd parties cant make your tx score worse, only better
+
+- When attaching more children, the parent tx can only get better score (until cluster limit is reached)
+
+Optimal Sort vs Ancestor Sort
+
+- maybe differently sized clusters get handed differently
+- maybe miners optimize how they sort clusters better than relay nodes
+
+Could there be a problem introducing non-determinism to the network?
+
+- Are we going to break more stuff?
+- The non-determinism doesn't matter, what matters is the baseline of what we guarantee:
+- For example, we run the ancestor algorithm always, and then sometimes we do more
+- Then no real use case can rely on anything higher than that bar
+
+We had a hard time coming up with examples in which our mining algorithms don't do the right thing.
+
+Maybe we can even relay sorted clusters, maybe someone has a better sort than you, let’s share it
+
+A new tx comes in, we could just throw it on the end of its cluster and then re-linearize later. i.e. have multiple linearizations for same cluster, then merge them together.
+
+## Open Questions III
+
+- We take the worst chunk across all clusters and evict that chunk.
+- "free relay problem" - if you can evict a chunk using only one new incoming tx, that’s bad.
+- Maybe we need a chunk size limit and a cluster size limit
+- We still need ancestor size limit currently its 101 kvb?
+
+## Future Work
+
+- Solve cluster size limits
+- Downstream effects: positive impact on package relay and package validation
+- Maybe we need to reconsider fee estimation overall (fee estimation is broken because of CPFP)
+
+## DEMO time
+
+y axis: total cumulative fee
+x axis: total vBytes
+"lol someones paying huge fees right now, messing up the chart!""
+
+`getblocktemplate` is much faster with this

bitcoin-core-dev-tech/2023-04-25-package-relay-primer.md

@@ -0,0 +1,178 @@
+---
+title: Package Relay Primer
+categories: ['core-dev-tech']
+tags: ['bitcoin core', 'package relay', 'p2p']
+date: 2023-04-25
+speakers: ['Gloria Zhao']
+---
+
+Slides: <https://docs.google.com/presentation/d/12YPlmmaCiNNL83b3FDmYwa7FKHP7yD0krzkEzM-tkTM>
+
+## Problems
+
+## CPFP Doesn’t Work When Mempool Min Feerate Rises
+
+Bad for users who want to use CPFP and L2s, but also a glaring limitation in our ability to assess transaction incentive compatibility
+
+## Pinning
+
+- being able to feebump transaction is a pinning concern
+- counterpart can intentionally censor your transactions, and in L2 that can mean stealing your money because you didn’t meet the timelock
+
+### Pinning examples
+
+1. ANYONECANPAY -> anyonecanpin: add input that comes from huge unconfirmed low-fee rate tx -> absolute feerate needs to increase, so you're paying more fees for the tx to confirm slower
+2. shared descendant limit can be monopolized
+   1. descendant limit is 25 txs, so someone else can fill up that limit
+   1. we have a carve-out for CPFP already to try and mitigate that
+3. RBF Rule 3 is gameable
+   1. replacement fees must > all descendants, and they may be large, low feerate -> expensive
+
+- most L2s have pinning problems, so we need to fix that
+
+## Get rid of txid-based relay
+
+- we want to avoid txid based relay
+  - can't deduplicate txid and wtxid that correspond to the same tx
+  - can't deduplicate txs that only differ in witness
+- we still require txid-based relay for orphans, because txs specify prevouts by txid -> so we don't know which wtxid to request
+
+## Definitions
+
+- pinning attack: a censorship attack on relay/mempool level, abusing policy
+  - avoid getting into mempools
+  - staying in mempool but never getting mined
+  - a pinning attack is NOT paying more to get mined (even though it might be an attacker doing it, that's just a fee-based market)
+
+- package: any list of transactions that can be represented as a connected DAG
+- ancestor package: a package of 1 tx and its (unconfirmed) ancestors
+- Descendant package: (fill in from slides)
+- Package relay: relaying and validating packages together
+- Sender-initiated: a node proactively announces packages they think their peers should download and validate together
+  - in initial proposal: suggested announcing a child with all of its parents if assumed that peer does not have parents yet (e.g. becausee parents are toow low fee)
+- Receiver-initiated: nodes can request packages when they recognize they're missing something
+
+## Why is there so much mempool code?
+
+- Peers are not trusted to provide correct information.
+
+## Sub-projects and problems they solve
+
+- Package CPFP: mempool logic to allow descendants to allow for ancestors. Accept packages, allow a child to bump a parent past mempool min feerate. Solves "CPFP doesn't work when mempools are full problem"
+- P2P Package Relay: additional protocol msgs to request, provide, download package information on p2p network.
+- Package RBF: also allow a child to pay for parent's conflicts (treat as 1 aggregated tx). However, painful pinning attacks still exist.
+- v3 policy: for things that want robust RBF
+  - make it feasible for them to be 0-fee without introducing dos vectors
+- ephemeral anchors (built on top of v3) which allows anchors to be 0-value, which allows us to remove need for CPFP carve-out
+
+## Progress so far
+
+- we have:
+  - package cpfp
+- open:
+  - v3
+  - [#29633](https://github.com/bitcoin/bitcoin/pulls/29633) (looking for review)
+  - [#27463](https://github.com/bitcoin/bitcoin/issues/27463): overview of package relay
+    - also have a full branch for package relay
+
+## Open questions exist
+
+- how to make orphanage robust enough, currently 3 different approaches
+- make a new mempool.dat file?
+- do we want splice in/out to be covered by package relay/v3?
+  - yes, but need to look into whether current approach is sufficient
+
+## Walking through the BIP
+
+- `ancpkginfo` provides a transaction's ancestors
+- `getpkgtxns` allows the receiver to request any subset of the announced txs
+- shows how Package Relay fixes CPFP. does not rely on feefilters as long as you don't reject a
+  package because it contains a low fee tx.
+
+Should there be a limit on the number of transactions in getpkgtxns? Should there be a limit on the
+number of transactions in ancpkginfo?
+
+- we already bound the network message size (to 4M bytes of payload)
+- Could we ever generate a message that exceeds the limit?
+
+`pkgtxns` should use the same order as `getpkgtxns`
+
+`ancpkginfo` SHOULD sort topologically and include all unconfirmed ancestors and the tx itself.
+However we cannot enforce this without matching chainstate.
+
+The combined hash is lexicographic, so the same set of transactinos will always have the same identifier
+
+`sendpackages` should perhaps specify how big an ancestor package would get accepted
+alternatively, it should be left up to the receiver to deal with ancestor packages that are too large.
+
+Looking for a different term for `pchCommand`
+
+Q: How far is the pullrequest?
+
+A: Tracking issue: <https://github.com/bitcoin/bitcoin/issues/27463>
+some of the earlier pieces are opened as PRs. persisting ancestor sets over restarts, p2p messages to negotiate package relay, handling orphans more reliably. There are
+about 5 more functional milestones to the whole project. One or two milestones per release seems reasonable.
+
+Orphan handling should be made more robust, e.g. we request parent information from any peer that announced an orphan, and prefer outbound peers as the source.
+
+Q: Could the P2P changes maybe all be activated on Signet together for testing?
+
+A: Yes, there is a branch with the whole implementation already.
+Unclear who needs it for testing, signet has a workaround (pseudo ephemeral anchors uses
+prioritisetransaction) in the absence of package relay.
+
+Mempool, Validation, Policy could be opened for merging in parallel to Orphan handling since different reviewers would be required.
+
+The activation of the package relay feature would set a config option to default false which only would get set to true at the end of the last milestone.
+
+Q: How does this interact with V3?
+
+A: V3 would follow the activation of package relay.
+
+Going into the Milestones:
+
+## Milestone 1
+
+1. "Don’t allow anything below min relay feerate (#26933)" was merged today. We generally don’t allow transactions with an individual feerate below minRelayTxFee, but V3 will permit 0-fee txs as the parent, and CPFP can be used to get past the dynamic mempool minimum feerate.
+
+2. "Persist CPFP’d transactinos across restarts (#27476)" Loading mempool.dat does not enforce minimum feerate on loading, but afterwards trims the mempool to the permitted size.
+
+- lots of discussion on importance and viability of the approach, PR to be deprioritized in the stack of work.
+- possibly just amend the mempool.dat format that can store package relationships
+- Just don’t modify `TrimToSize()`
+
+3. validate package transactions with their in-package ancestor sets (#26711)
+
+- want to be able to handle something more complicated than single-parent-single-child
+- Allow any ancestor package
+- Be lenient on what you’re provided with, e.g. take valid subsets of the package instead of rejecting the whole package if a part is inacceptable.
+- find largest permitted subset
+  – We defer Package RBF for the moment
+
+## Milestone 2
+
+- Orphanage is currently limited to 100 txs, up to 400 kB each
+- Problem: Peer can churn orphanage by sending a ton of orphans
+- Problem: does not effectively protect memory
+- We randomly evict when orphanage overflows, but we might not be anywhere close to the theoretical max
+
+Idea 1: only protect one package per peer, or have a huge potential upper limit
+
+- terrible performance, tons of packages are gonna be dropped
+
+Idea 2: Hard limit globally to a number of bytes, when you go over limit, evict things from peer that has sent more than 2*maxTxSize
+
+Idea 3: Token bucket per peer to limit exposure to bad packages
+
+Idea 2 gets the most positive feedback.
+
+## Milestone 3
+
+Add TxPackageTracker
+
+Interface vision. This will be more interesting to discuss when looking at the code.
+
+Rejection Caching
+
+We need to continue downloading ancestor package information because someone could give you invalid information that could be superseded by correct information.
+However, by keeping track of what we have gotten before, we would notice when we are offered the same invalid package or something that depends on something that was relayed previously.