wip: feat(txpool): new txpool design

[mattsse]

WIP

This is another txpool iteration that replaces the previous graph based pool with a tier-pool design described here https://github.com/ledgerwatch/erigon/wiki/Transaction-Pool-Design#design-proposal

The previous pool design, essentially was only able to handle the nonce gap condition. Introducing additional state-related conditions turned out to be tricky.

Instead of two (pending, queued) sub-pools, this has three subpools (pending, basefee, queued) and can also handle: dynamic fee + balance related restrictions.

The state of a tx is maintained in a bitfield from which the subpool the tx should be moved to is derived.

There's only one BTreeMap((Sender,Nonce), Tx) that tracks all transactions in the pool. The subpools then contain a reference (or rather an Arc clone) to the transaction.

By storing all transactions in one place, state changes can be derived more efficiently than checking separate pools.
By sorting by (Sender,Nonce) all descendant tx of a tx can be visited efficiently, this is imported to update the state of transactions from the same sender.

If a new tx is added, or a block was mined, appropriate updates are derived which are then applied to the subpools (e.g.: move tx from queued to ready).

The pending pool returns an iterator (BestTransactions) that always returns the next best transaction.

todo:

• simplify tx/pool types
• handle more conditions
• ...

[mattsse]

@rakita wdyt about these changes?

one thing I think needs to be changed is the Priority type.

I guess we could instead make this a generic compare(tx, tx) -> Ordering function instead? which is used to determine best/worst in the queued/basefee pool, so we can pop worst if we exceed size limits.

And for pending I think, we need a sort function that can rank a transaction in relation to other tx in the pool. This is a more advanced use case and maybe we should use compare here as well for now and revisit later.

I kept the internal mapping for Address -> u64 since this is used heavily in the total BtreeMap of all transactions.

[gakonst]

As discussed, trust you to design this according to Erigon & the patterns for multi-tx pool, let's make sure to add a diagram later.

[rakita]

Priority seems okay for this level of abstraction, as you can put anything that you want to compare inside it. And it has Ord.

for anything you would do inside compare(tx, tx) -> Ordering as in tx1.price < tx1.price you are extracting data to Priority= tx.price and comparing price after that price1 < price2 it is same principle with having compare(tx,tx)

Maybe it would be a cleaner implementation but as I can tell you are getting similar functionality.

[rakita]

Could you use timestamp from ValidPoolTransaction for this?

[mattsse]

I've thought about this as well, but since Instant depends on the OS this could lead to issues. I saw weird timestamps in the past, especially after reboot.

[rakita]

Instant can report time in past sometimes (few ns or very small margin), and panic would happen if you sub them with that assumption (had a bug with this :) ), for txpool where transactions come in arbitrary order in the span of seconds/minutes we just need a comparison we wouldn't care about value.

[mattsse]

if you sub them with that assumption (had a bug with this :)

same :D

makes sense, I'll use the timestamp instead of the id then

[rakita]

Maybe we need to define (broadly) what flexibility/functionality and use case we want to have in txpool, we could over abstract and have not so useful complexity if we are not sure how it is going to be used.

lgtm

[mattsse]

sg!

I wrote additional tests, will merge this as is, and then start smaller iterations. starting with docs + definition

[gakonst]

Maybe we need to define (broadly) what flexibility/functionality and use case we want to have in txpool, we could over abstract and have not so useful complexity if we are not sure how it is going to be used.

Yeah let's start being explicit about what it provides, can provide in the future, and any limitations.