-
Notifications
You must be signed in to change notification settings - Fork 20.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add hashcache to pbss difflayer #29991
base: master
Are you sure you want to change the base?
Conversation
Hmmm, interesting idea with using the hash as the cache key since it can't change, so you don't need to track liveness across layers. Out of curiosity, wouldn't using an existing cache type work instead of defining your own? We use (I think) victoriametrics.
|
The performance of using fastcache is basically the same, but it takes up extra memory~ |
Seems your PR has a consensus fault in it, PTAL: https://ci.appveyor.com/project/ethereum/go-ethereum/builds/50010174 |
@karalabe this pull request is not compatible with verkle. I guess it might be the reason for the failing test. |
Fixed~ |
how much performance gain we can have by applying it to state snapshot. |
cache: make(map[common.Hash]*RefTrieNode), | ||
} | ||
case *diffLayer: | ||
dl.origin = l.originDiskLayer() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here you can get the parent origin directly without the read lock, because it's already gotten when it's called.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Semantically, the origin disklayer is mutable, and perhaps the read and write operations of the origin can also be serialized, without the need for a lock guard. But adding a lock guard may be clearer :)
At present, adding multi-version cache to snapshot difflayer does not improve performance. The main reasons are as follows:
|
Description
In the native transfer test scenario, we found that the bottleneck of pbss trienode reading is the recursive query of the 128-layer difflayer. The current PR is mainly to optimize the pbss trienode reading latency.
Rationale
In difflayer, a new cache map with node hash as key is added. The Node interface gives priority to accessing the cache. If a hit is found, it is returned directly. If it is not found, the disklayer is queried. Compared with the recursive query of the 128-layer difflayer, this cache query is O(1). The original query path is optimized, and most of them are fastpath.
The cache is added when the difflayer is added to the layertree and is released when the difflayer is merged into the disklayer.
Performance
Test scenario: 1 million accounts were randomly selected from 25 million accounts, and native transfer transactions were performed at 600qps. The difflayer cache optimization effect was significant, and the validation phase was reduced from 450ms to 100ms.
Changes
Notable changes: