[WIP]Separate Processing and State Verification on BSC: implement the framework of fast node

[realuncle]

Summary

I am a developer from https://nodereal.io/ who focuses on building stable and powerful infrastructure in the cryptocurrency world.

This PR tries to improve the syncing speed by separating the block processing and state verification in two different nodes: fast node and verify node.

Abstract

Currently a BSC node has two kinds of state world: MPT and snapshot. MPT(Merkle Patricia Tries) is a tree-structured state world. The key function of MPT is to generate the state root to ensure state consistency, while the query/commit on MPT is quite slow. Snapshot is a flattened key-value-based state world. Snapshot provides fast queries and commit. The storage size of the snapshot increases slowly even with a large transaction volume. Snapshot is usually used for block processing, while MPT is used for state verification.

To make full use of different storage, the fast node will do block processing with snapshot. It will do all verification against blocks except state root. The verify node receives the difflayer from fast node and does state verification, and response to fast node. All these are nonblocking process so that the syncing can be fast.

Motivation

The increasing adoption of BSC leads to a more active network. On the other hand, the node maintainer had a hard time keeping their node catching up with the chain. A new syncing protocol to lower the hardware requirement is an urgent need.

Specification

Fast node is a bsc client that does fullsync using only snapshot and generate difflayer. It needs the confirm message from the verify node before freezing the blocks, it has to wait if 1. it receives any negative message from the verify node; 2. it does not receive a positive message from the verified node for blocks at height CurrentHeight-FreezeNumber.

Verify node receive difflayer that is generated by fast node and try to apply it into MPT and verify the state root. It sends back positive/negative messages to the fast node.

All the messages exchanged between Fast node and Verify node are based on the diff p2p protocol.

Authentication

Fast node can only rely on trusted verify node, either deployed by the same developer or deployed by a trusted organization. We know the different peers will verify the peer id during the handshake, we will borrow this mechanism to do the authentication. We introduce TrustVerifyNodes settings, which is a list of encoded addresses, the fast nodes only build connections(based on diff protocol) with TrustVerifyNodes.

For organizations, they can deploy their own verified node.

For individual developers, they can connect to the verify node donated by the famous organization/validators.

Chain Tools

Implement a new prune command to prune all MPT storage.

Prototype verification

The performance improves x3 in fast node on mainnet.

Command

Add --allow-insecure-no-tries to enable the feature, like ./geth --config ./config.toml --datadir ./node --syncmode full --cache 5000 --allow-insecure-no-tries

Prune the tires node: ./geth snapshot insecure-prune-all --datadir ./node ./genesis.json

[RenRick]

LGTM

[iakisme]

./geth snapshot insecue-prune-all --datadir ./node ./genesis.json

replace(insecure-prune-all,insecue-prune-all)

[ghost]

change