Multithreaded mempool

[sidhujag]

Big innovation update: Multi-threaded transaction processing through the mempool.

This theoretically uncaps the constraint of traditional bitcoin style blockchains which rely single threaded mempool inclusions for security reasons. The main reason is to avoid DDOS attacks from bad signature transactions which would relay across the network at minimal cost to wreak havoc on the p2p layer of full nodes. We got around this by creating a policy whereby if a bad signature verification is detected we fallback to single threaded mode for 60 seconds. This means that:

1. Every 60 seconds a tx with verification failure is permitted such that it will not ban the sender node but subsequent requests in single threaded will follow the strict policy that will check signature verification and ban if they fail(on 3 signature validation failures from that node). An attack would consist of someone sending messages once every 60 seconds and hope that they do not get banned, if they send more than 4 per 60 seconds they will likely start to get banned across the network which is what we want to see. More heuristics can be applied as we profile the system working as intended.

2. In multithreaded mode, the verification happens in a separate threadpool (number of threads determined by the number of cores, so if your on a quadcore CPU with hyperthreading you will get 8 threads total (# of cores * 2 because of hyperthreading optimizations). This optimizes the amount of work you can do at any given time in a parallel mechanism instead of single core and wasting precious resources of full nodes.

3. In multithreaded mode, since the verification happens in separate threads, the main thread is free to relay the transaction in real-time likely faster than the verification will happen. This means that the relay is not dependent on the verification bottleneck. Transactions will be relayed as fast as possible across the network with no regard to the ECDSA checks. Of course if the check fails the nodes always fallback to single threaded mode for 60 seconds and also will enable the ban policy to act against the sending peer on subsequent failures just as bitcoin does.

Theoretically the bottleneck is put back onto the network stack and overall the size of the mempool for the long running TPS tests. Overall with 10mb every minute and 300 bytes on average for one assetallocationsend via ZDAG the max sustained longterm rate is 555 tx/s but burst tests are now limited by network propogation.

Network progogation is optimized by masternodes by providing a better topology than bitcoin because of the full node incentive program through the bonding validator mechanism. Each masternode can be connected to 25 or more peers which creates a fat layer of full nodes that will likely get to a receiving peer of the asset in less network hops and thus less network latency between a sender and receiver than bitcoin or the like. Majority of the fullnodes because of these incentives are these masternodes with high connection counts and dedicated hardware ready to relay transactions and verify in parallel.

On a 1 gb network connection which is typical amongst non-entry level VPS's we can achieve 123MB/s throughput which equates to about 7000 transactions @ 300 bytes per second sustainable. As we increase up the network link and the number of cores per masternode we can effectively scale up our network throughput of ZDAG on traditional off-the-shelf hardware.

[phj5]

Looks great! This multi-threading verification approach incorporated with relay feedback rules is eliminating one of the biggest TPS bottlenecks, tx verification. Also this will allow us to minimize the attack surface on the node relay at the same time.