feat: Implement HADS using single RocksDB

[fominok]

This PR implements HADS structure using dense Merkle tree (rs-merkle) as a root tree and merkelized AVL tree (Merk) as any other subtree. This PR doesn't include proofs generation.

As it contains a lot of changes I'll try to describe how it works to verify that we're on the same page.
Note that GroveDB methods expects bytes as paths, keys and values, this example uses strings for clarity.

0. GroveDB structure

First, here is a description of GroveDB internal state:

• Dense Merkle tree as a root tree (root_tree)
• Reference to RocksDB connection (db)
• Map of subtree prefix* to Merk object (subtrees)
• Map of subtree prefix to index in the root tree (root_leaf_keys)

* subtree prefix is a concatenated path of a subtree, like if we had path == [b"aa", b"bb", b"cc"] it will be prefix == b"aabbcc"; then each key for a Merk will be prefixed with this prefix before insertion to RocksDB, retrieval works the same way, so that's how we're able to access a proper Merk object by path and save/restore multiple Merks using single RocksDB.

Next we'll apply some changed to GroveDB to see how its state is changed.

1. Insert two subtrees into the root tree

let mut db = GroveDb::open("db")?;
db.insert(&[], "key1", Element::empty_tree())?;
db.insert(&[], "key2", Element::empty_tree())?;

This will result into the following state:

1. Root tree (leafs listed by index)
   • 0: subtrees[b"key1"].root_hash()
   • 1: subtrees[b"key2"].root_hash()
2. subtrees map
   • b"key1": Merk::open(...) // prefix is b"key1"
   • b"key2": Merk::open(...) // prefix is b"key2"
3. root_leaf_keys map
   • b"key1": 0
   • b"key2": 1
4. RocksDB keys:
   • SUBTREES_SERIALIZED_KEY: serialize(subtrees.keys()) // this stores all used prefixes into RocksDB to restore later
   • ROOT_LEAFS_SERIALIZED_KEY: serialize(root_leaf_keys) // this stores top level subtrees' positions to rebuild root tree later

2. Insert a subtree into subtree

db.insert(&[b"key1"], "b"key11", Element::empty_tree())?; 

This will result into the following state:

1. Root tree has the same structure, but recomputed, as "key1" subtree root hash has changed
2. subtrees map
   • b"key1": Merk::open(...) // prefix is b"key1"
   • b"key2": Merk::open(...) // prefix is b"key2"
   • b"key1key11": Merk::open(...) // prefix is b"key1key11"
3. root_leaf_keys map is unchanged
4. RocksDB keys:
   • SUBTREES_SERIALIZED_KEY: serialize(subtrees.keys())
   • ROOT_LEAFS_SERIALIZED_KEY: serialize(root_leaf_keys)
   • b"key1key11": serialize(Element::Tree(subtrees[b"key1key11"].root_hash())) // prefix is "key1" and key is "key11", their concatenation is a RocksDB key to store an element which wraps a root hash of a sub-sub tree

3. Insert a value into a sub-sub tree

This example should finally shed some light on how prefixes work.

db.insert(&[b"key1", b"key11"], b"key_final", Element::Value(b"ayy"))?;

This will result into the following state:

1. Root tree has the same structure, but recomputed, as "key1" subtree root hash has changed
2. subtrees map is unchanged
3. root_leaf_keys map is unchanged
4. RocksDB keys:
   • SUBTREES_SERIALIZED_KEY: serialize(subtrees.keys())
   • ROOT_LEAFS_SERIALIZED_KEY: serialize(root_leaf_keys)
   • b"key1key11": serialize(Element::Tree(subtrees[b"key1key11"].root_hash())) // recomputed as Merk was changed
   • b"key1key11key_final": serialize(Element::Value(b"ayy")) // subtree prefix is "key1key11", key is "key_final", their concatenation is RocksDB key

P.S. Please ignore restorer and crash_merk modules as they're temporary disabled

[QuantumExplorer]

why disable this test?

[QuantumExplorer]

I guess getting checkpointing working will come later?