New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: make txpool operate on immutable state #15085

merged 1 commit into from Sep 5, 2017


None yet
3 participants

karalabe commented Sep 4, 2017

Apparently the transaction pool (and handling in general) had 3 different update pathways:

  • On chain head events (and most notably reorgs), the pool's internal state was reset to the current chain state (and transactions filtered accordingly).
  • On transaction removals events (caused by reorgs), the old transactions were rescheduled.
  • On miner invalidations, transactions we're yet again concurrently dropped from the pool.

Unfortunately, the above is a very very bad idea from multiple perspectives:

  • During reorgs, the pool absolutely surely goes out of sync with itself, because its tracked nonces are updated based on head events, but its tracked transactions are updated based on removal events. These by their very nature happen concurrently, so any interaction with the transaction pool in between is undefined.
  • The exact same data race happens within the miner too, which was "hacked" to handle it by detecting these "strange" out-of-sync transactions and explicitly removing them from the transaction pool. However this again was a very bad idea, because now there's a third event racing to update the pool, which plays horribly with the reorg corruption window (and happens in the exact same scenario).

This PR tries to fix all these races via reducing the transaction pool's API surface to one single entry point: chain head events.

  • When a chain head event is received, we reset the pool's internal nonces to the current state as currently, but afterwards we gather the reorged transactions within the pool itself and apply any diffs before releasing the lock. This ensures that nonces, states and transactions are updates atomically and cannot go out of sync between each other.
  • A direct extension of the above is that all method calls from the transaction pool that accessed the blockchain/statedb/gaslimit directly is now gone, and instead only the chain head header is used to derive all needed fields (and StateAt(head.Root)). This ensures that even if the blockchain changes concurrently with the transaction pool, no changes leak into the pool without an isolated head event.
  • Lastly, the functionality to remove transactions from the pool from the outside was altogether removed. The miner was made a bit smarter so it can differentiate between a few transaction processing errors that it can safely ignore because the pool will filter on it own (nonce too low/high). This allows the last update path that poked holes into the transaction pool to be removed.

@ethereum ethereum deleted a comment from GitCop Sep 4, 2017

@ethereum ethereum deleted a comment from GitCop Sep 4, 2017

@ethereum ethereum deleted a comment from GitCop Sep 4, 2017

@ethereum ethereum deleted a comment from GitCop Sep 4, 2017

@karalabe karalabe added this to the 1.7.0 milestone Sep 4, 2017

@karalabe karalabe requested review from fjl and holiman Sep 4, 2017

Show outdated Hide outdated core/tx_pool.go

holiman approved these changes Sep 5, 2017


Show outdated Hide outdated core/tx_pool.go
Show outdated Hide outdated core/tx_pool.go
Show outdated Hide outdated core/tx_pool.go

@karalabe karalabe merged commit c91f7be into ethereum:master Sep 5, 2017

1 of 3 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
continuous-integration/travis-ci/pr The Travis CI build is in progress
commit-message-check/gitcop All commit messages are valid
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment