You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be a nice feature for the application layer not to have to worry about normalising data that may get stored twice. For example in the case of Burrow things like deployed bytecode or serialised transactions stored multiple times in events could be normalised in the application layer but it is more general if we can lean on the storage layer for this.
The way this would work is value []byte in Node would become valueAddress []byte where valueAddress is the hash of the value. When writing a value we would first try and read the value from the db by its hash, if the key is found then no write is performed. We could also add a cache similar to nodeCache for values that could catch reads (at least for nodes in the nodeCache).
In this way we would deduplicate storage across the lifetime of the chain - depending on the level of duplication this could be significant. Also, knowing this, applications can try to avoid mixing noise with values that might otherwise be dedupable when storing in the tree. It would cost almost nothing computationally given that we are already hashing Node each time we save - this would just make that a two step process where we first hash the value and then only have that hash to hash for the node.
I wanted to get some feedback on this, but if I actually get some time to spend on the state subsystem of Burrow this is something I'd be interested in taking on. Though I realise I have been threatening to do something useful round here for a while ... :o
The text was updated successfully, but these errors were encountered:
It would be a nice feature for the application layer not to have to worry about normalising data that may get stored twice. For example in the case of Burrow things like deployed bytecode or serialised transactions stored multiple times in events could be normalised in the application layer but it is more general if we can lean on the storage layer for this.
The way this would work is
value []byte
inNode
would becomevalueAddress []byte
wherevalueAddress
is the hash of the value. When writing a value we would first try and read the value from the db by its hash, if the key is found then no write is performed. We could also add a cache similar tonodeCache
for values that could catch reads (at least for nodes in thenodeCache
).In this way we would deduplicate storage across the lifetime of the chain - depending on the level of duplication this could be significant. Also, knowing this, applications can try to avoid mixing noise with values that might otherwise be dedupable when storing in the tree. It would cost almost nothing computationally given that we are already hashing
Node
each time we save - this would just make that a two step process where we first hash the value and then only have that hash to hash for the node.I wanted to get some feedback on this, but if I actually get some time to spend on the state subsystem of Burrow this is something I'd be interested in taking on. Though I realise I have been threatening to do something useful round here for a while ... :o
The text was updated successfully, but these errors were encountered: