Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider content-addressing values in the nodeDB #64

Closed
silasdavis opened this issue Jun 18, 2018 · 0 comments
Closed

Consider content-addressing values in the nodeDB #64

silasdavis opened this issue Jun 18, 2018 · 0 comments

Comments

@silasdavis
Copy link
Contributor

silasdavis commented Jun 18, 2018

It would be a nice feature for the application layer not to have to worry about normalising data that may get stored twice. For example in the case of Burrow things like deployed bytecode or serialised transactions stored multiple times in events could be normalised in the application layer but it is more general if we can lean on the storage layer for this.

The way this would work is value []byte in Node would become valueAddress []byte where valueAddress is the hash of the value. When writing a value we would first try and read the value from the db by its hash, if the key is found then no write is performed. We could also add a cache similar to nodeCache for values that could catch reads (at least for nodes in the nodeCache).

In this way we would deduplicate storage across the lifetime of the chain - depending on the level of duplication this could be significant. Also, knowing this, applications can try to avoid mixing noise with values that might otherwise be dedupable when storing in the tree. It would cost almost nothing computationally given that we are already hashing Node each time we save - this would just make that a two step process where we first hash the value and then only have that hash to hash for the node.

I wanted to get some feedback on this, but if I actually get some time to spend on the state subsystem of Burrow this is something I'd be interested in taking on. Though I realise I have been threatening to do something useful round here for a while ... :o

@tac0turtle tac0turtle closed this as not planned Won't fix, can't repro, duplicate, stale Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants