Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: concurrent background transaction sender ecrecover #16882

Merged
merged 1 commit into from Jun 5, 2018

Conversation

Projects
None yet
2 participants
@karalabe
Copy link
Member

karalabe commented Jun 4, 2018

A single ecrecover operation on my laptop (Core i7) takes approximately 165us according to our benchmarks (can go up to 220us if the machine is hot and CPU gets throttled).

$ go test --bench=Ec ./crypto/
BenchmarkEcrecoverSignature-8   	   10000	    164167 ns/op

That seems fairly low and insignificant (5000-6000 ecrecovers / second), until we realize there are 243M transactions on mainnet... which takes approximately 12-14 hours to ecrecover on a single thread if running a full sync.

Post sync block propagation also suffers on mainnet when we consider that a block has about 250 transactions in them. That amounts to about 50ms of processing time just for recovering the senders from the transactions.

Uncles further worsen the issue, because they usually arrive at the same time with the canonical block, so they incur an extra 50ms processing hit. Furthermore any transaction reorged out of the canonical chain is loaded back into the pool, incurring ecrevocer costs once more.


A functionally correct solution would be to introduce ecrecover threading to all places where we might see performance gains, and properly synchronize and process the results similar to PoW checks. Unfortunately that would be way too expensive and messy, because:

  • Sync needs to concurrently ecrecover across many blocks (some potentially empty).
  • Propagation needs to concurrently ecrecover within a single block.
  • Transaction reorg needs to ecrecover random sets of txs.

Launching goroutines when there isn't enough work would outweigh the benefits, waiting for the results could introduce blockages and unneeded sync overhead. And overall complexity would easily become prohibitive.

Luckily our transaction objects support caching the senders for later retrieval. This means that we don't need to implement fancy scheduling to ensure the recoveries are correct but still fast. Rather we can create the fastest and simplest way to ecrecover the signers, and any occasional hiccups (didn't recover in time when needed, recovered with the wrong signer during a fork block, etc) auto correct themselves.

This PR solves the concurrent ecrecovery task by launching a numcpu pool of global recovery threads, each waiting for transaction lists to recover. Whenever a list of transaction is in need of recovery, it is split between all the workers, which execute the operations and cache the results back into the origin transaction. To maximize performance the same underlying buffer is sent to all workers, but not in contiguous chunks, rather strided (thread 1 processes tx 1, 1+N, 1+2N, etc).

Recovery tasks are pushed by the blockchain and the tx pool during block importing and tx rescheduling. This ensures that we relieve sync/propagation ecrecover as well as reorg ecrecover slowdowns.


Performance stats:

Block master ecrecover
4.234M 17h 40m 14h 40m
5.745M 95h 59m 77h 45m

@karalabe karalabe requested a review from holiman as a code owner Jun 4, 2018

@karalabe karalabe force-pushed the karalabe:streaming-ecrecover branch from f931b2f to 2ab24a2 Jun 5, 2018

@karalabe karalabe changed the title [wip] core: concurrent background transaction sender ecrecover core: concurrent background transaction sender ecrecover Jun 5, 2018

@holiman

holiman approved these changes Jun 5, 2018

Copy link
Contributor

holiman left a comment

LGTM

@karalabe karalabe added this to the 1.8.10 milestone Jun 5, 2018

@karalabe karalabe merged commit 01a7e26 into ethereum:master Jun 5, 2018

2 of 3 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
commit-message-check/gitcop All commit messages are valid
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@karalabe karalabe modified the milestones: 1.8.10, 1.8.11 Jun 11, 2018

jpeletier pushed a commit to epiclabs-io/go-ethereum that referenced this pull request Jun 14, 2018

Merge pull request ethereum#16882 from karalabe/streaming-ecrecover
core: concurrent background transaction sender ecrecover
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.