forked from bitcoin/bitcoin
-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multithreaded mempool #210
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Syscoin 3.0.4
if verification fails within the last 60 seconds then flip back to single threaded
also add scriptexectioncache once multithread checks are done
phj5
approved these changes
May 29, 2018
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! This multi-threading verification approach incorporated with relay feedback rules is eliminating one of the biggest TPS bottlenecks, tx verification. Also this will allow us to minimize the attack surface on the node relay at the same time.
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Big innovation update: Multi-threaded transaction processing through the mempool.
This theoretically uncaps the constraint of traditional bitcoin style blockchains which rely single threaded mempool inclusions for security reasons. The main reason is to avoid DDOS attacks from bad signature transactions which would relay across the network at minimal cost to wreak havoc on the p2p layer of full nodes. We got around this by creating a policy whereby if a bad signature verification is detected we fallback to single threaded mode for 60 seconds. This means that:
Every 60 seconds a tx with verification failure is permitted such that it will not ban the sender node but subsequent requests in single threaded will follow the strict policy that will check signature verification and ban if they fail(on 3 signature validation failures from that node). An attack would consist of someone sending messages once every 60 seconds and hope that they do not get banned, if they send more than 4 per 60 seconds they will likely start to get banned across the network which is what we want to see. More heuristics can be applied as we profile the system working as intended.
In multithreaded mode, the verification happens in a separate threadpool (number of threads determined by the number of cores, so if your on a quadcore CPU with hyperthreading you will get 8 threads total (# of cores * 2 because of hyperthreading optimizations). This optimizes the amount of work you can do at any given time in a parallel mechanism instead of single core and wasting precious resources of full nodes.
In multithreaded mode, since the verification happens in separate threads, the main thread is free to relay the transaction in real-time likely faster than the verification will happen. This means that the relay is not dependent on the verification bottleneck. Transactions will be relayed as fast as possible across the network with no regard to the ECDSA checks. Of course if the check fails the nodes always fallback to single threaded mode for 60 seconds and also will enable the ban policy to act against the sending peer on subsequent failures just as bitcoin does.
Theoretically the bottleneck is put back onto the network stack and overall the size of the mempool for the long running TPS tests. Overall with 10mb every minute and 300 bytes on average for one assetallocationsend via ZDAG the max sustained longterm rate is 555 tx/s but burst tests are now limited by network propogation.
Network progogation is optimized by masternodes by providing a better topology than bitcoin because of the full node incentive program through the bonding validator mechanism. Each masternode can be connected to 25 or more peers which creates a fat layer of full nodes that will likely get to a receiving peer of the asset in less network hops and thus less network latency between a sender and receiver than bitcoin or the like. Majority of the fullnodes because of these incentives are these masternodes with high connection counts and dedicated hardware ready to relay transactions and verify in parallel.
On a 1 gb network connection which is typical amongst non-entry level VPS's we can achieve 123MB/s throughput which equates to about 7000 transactions @ 300 bytes per second sustainable. As we increase up the network link and the number of cores per masternode we can effectively scale up our network throughput of ZDAG on traditional off-the-shelf hardware.