Use lazy-loaded cached tree implementation #6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This introduces a largely rewritten implementation of the SMT which operates on a cached, lazily-loaded tree structure which must be explicitly committed to DB, replacing the existing pattern of performing DB reads and writes on for each operation. This gives a significant performance improvement on all operations while preserving backwards compatibility of commitment hashes. This also: * refactors the hasher objects to allow tree paths and stored values to be hashed independently, and Options to configure them. By passing an identity function as the path hasher, the raw key can be used directly as the leaf. * removes value storage from the tree; the caller must maintain their own value mapping. * removes the DeepSubtree code supporting state-transition fraud proofs, as it was not compatible as-is with this implementation (but there should be no technical block to adapting it to this pattern). * renames various types and decouples the path and value hashing functions. * expands tests to cover corner cases, e.g. orphan node removal.
Olshansk
approved these changes
May 24, 2023
There was a problem hiding this comment.
The hashers.go file was missing from the repo and had to be copied in
Damn...
Going to approve this just so we can merge it in and start iterating on top of the lazy-loaded PR. A couple things to keep in mind (before we clean up the code and such):
- Can you try to integrate the lazy-loaded PR into the pocket repo and see if everything still works as expected?
- Note that the README in this repo will also need to be updated, but happy to leave that for a future PR as well.
With that said
9 tasks
|
Thanks for pulling from the new branch, and pointing out the missing file - sorry about that! |
No worries thanks for all your hard work! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a port of this PR whilst retaining the squashed commit history of @roysc
Although using the latest vulcanize/smt lazy_loading branch some changes had to be made:
hashers.gofile was missing from the repo and had to be copied incelestiaorg/smtwere replaced withpokt-network/smtThis solves and merge conflicts and enables lazy loading to be used in the repo.