Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage][execution] Switch to Jellyfish Merkle Tree #516

Merged
merged 1 commit into from Aug 12, 2019

Conversation

@lightmark
Copy link
Contributor

commented Aug 11, 2019

Motivation

We are ready to switch to jellyfish merkle tree with the following changes:

  1. Change all get_xxx_by_state_root apis to get_xxx_by_version;
  2. Add version to any method that requires it including execution module.
  3. Switch from SparseMerkleTree to JellyfishMerkleTree in storage.
  4. Update tree cache to include internal/leaf stats (new/stale).
  5. rename retired_record to stale_node_index.

Test Plan

cargo test

Related PRs

#424 #441

@lightmark lightmark requested review from msmouse, wqfish and zamsden Aug 11, 2019
@lightmark lightmark self-assigned this Aug 11, 2019
@lightmark lightmark force-pushed the lightmark:integration branch from cf28380 to da8949a Aug 11, 2019
@lightclient

This comment has been minimized.

Copy link

commented Aug 11, 2019

Hi @lightmark -- just curious, what is a jellyfish merkle tree? Not coming up with much from Google.

@lightmark lightmark force-pushed the lightmark:integration branch from da8949a to 45a38bf Aug 11, 2019
@lightmark

This comment has been minimized.

Copy link
Contributor Author

commented Aug 11, 2019

@lightclient it is a customized Patricia Sparse Merkle Trie designed by us. We call it Jellyfish because the tree finally will be like a titanic jellyfish with millions of tentacles(accounts). We will have doc/whitepaper later describing the details. For now you can read the code.

@lightclient

This comment has been minimized.

Copy link

commented Aug 11, 2019

@lightmark lightmark added the executor label Aug 12, 2019
execution/executor/src/block_processor.rs Outdated Show resolved Hide resolved
execution/executor/src/block_processor.rs Outdated Show resolved Hide resolved
storage/libradb/src/pruner/test.rs Show resolved Hide resolved
storage/libradb/src/state_store/mod.rs Outdated Show resolved Hide resolved
storage/libradb/src/state_store/mod.rs Outdated Show resolved Hide resolved
storage/storage_proto/src/lib.rs Outdated Show resolved Hide resolved
@msmouse

This comment has been minimized.

Copy link
Contributor

commented Aug 12, 2019

This looks like it's good to go!
It would be great if we could have a cleaner history for this PR, though. Would you mind squashing your commits in the best succinct organization before we merge?

@lightmark lightmark force-pushed the lightmark:integration branch 3 times, most recently from 7a8b2ce to f46f5a6 Aug 12, 2019
execution/executor/src/lib.rs Outdated Show resolved Hide resolved
@lightmark lightmark force-pushed the lightmark:integration branch from f46f5a6 to d0bdc97 Aug 12, 2019
@@ -46,12 +46,13 @@ impl MockTreeStore {
}

pub fn write_tree_update_batch(&self, batch: TreeUpdateBatch) -> Result<()> {
let (node_batch, retired_record_batch) = batch.into();

This comment has been minimized.

Copy link
@zamsden

zamsden Aug 12, 2019

Contributor

Do you need an .into() implementation anymore or was this the only use?

This comment has been minimized.

Copy link
@lightmark

lightmark Aug 12, 2019

Author Contributor

now the batch has 4 fields so I think it is better to avoid .into()

@lightmark lightmark merged commit 8321ac8 into libra:master Aug 12, 2019
1 check passed
1 check passed
commit-workflow Workflow: commit-workflow
Details
@lightmark lightmark deleted the lightmark:integration branch Aug 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.