This repository has been archived by the owner. It is now read-only.

Consensus/Blockchain size #2897

Closed
S-anasol opened this Issue Mar 21, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@S-anasol
Contributor

S-anasol commented Mar 21, 2018

Hi!

My question about fast consensus grow.

Of course not so big and high priority problem, but that's annoying.

Sia currently doesn't have so much transactions and contracts as it possibly can have in future.
So size growing with so little activity in network looks terrifying in perspective.

I already have "fixed" size problem few months ago at https://consensus.siahub.info/ with gz that gave me 50% disk space/bandwidth savings.
But anyway it have "uncomfortable" size for bootstrapping and usage for users.

We already have 150gb Bitcoin and other fat coins, doesn't look as good way to follow this trend :)

Is there any plans to optimize blockchain size? Is it even possible?

@lukechampine

This comment has been minimized.

Member

lukechampine commented Mar 21, 2018

One option would be to create a "checkpoint" or "snapshot" of the blockchain. The snapshot would simply be the set of unspent outputs as of some blockheight. I wrote a script to calculate the size of the resulting snapshot, and at the current blockheight (~146000) it estimates 300MB, or about 1/30th the size of the blockchain.

If you have a snapshot, you can perform consensus validation. However, you do not have access to blocks prior to the snapshot. So we would want to discourage people from using snapshots, since they hurt the overall security of the network.

Another perspective is that it's okay for the blockchain to be fairly large. Currently, it's a problem because you need to download the blockchain in order to rent or host -- but that won't always be the case. It's easy to imagine a large set of renters or hosts (e.g. an internal LAN) that all trust one full node.

@S-anasol

This comment has been minimized.

Contributor

S-anasol commented Mar 22, 2018

My thought actually about optimization of blockchain structure.
Not sure how exactly BoltDB store data to disk, but looks like there is huge overhead with all that Sia's JSON structures.
Maybe I wrong and it compresses blocks as possible and remove/replace any keys, to avoid storing full json for each block?

@lukechampine

This comment has been minimized.

Member

lukechampine commented Mar 23, 2018

The consensus db stores blocks in binary (using Sia encoding), not JSON. So there shouldn't be too much wasted space. Compression probably helps because Bolt allocates more space than it needs (to avoid reallocating too often).

@tbenz9

This comment has been minimized.

Collaborator

tbenz9 commented Mar 28, 2018

I'm going to close this as the original issue asked a question and the question has been answered. Feel free to reopen the issue if you think it was prematurely closed.

@tbenz9 tbenz9 closed this Mar 28, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.