New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: download the entire blockchain over bittorrent #14551

Open
Glamdring opened this Issue May 30, 2017 · 7 comments

Comments

Projects
None yet
7 participants
@Glamdring

Hello,

This is an improvement proposal, rather than a bug report. And I'm not a blockchain expert, so it might not be a great idea, but...

Currently I see a great usability issue in the fact that the blockchain is downloaded (synced) very slowly. For me it crashed a couple of times, which which in no way is helping adoption among not so technical users.

So why not push snapshots of the blockchain to a bittorrent DHT? People have done that with the Bitcoin blockchain - a snapshot is available for download from various places.

But manual download isn't ideal - my proposal is to include a small bittorrent client in geth (and other implementations), so that peers can use the ethereum protocol to push a particular snapshot to the DHT, and also "advertise" the same DHT key to newly joined peers.

I'm not sure if that can be secure enough, e.g. if malicious nodes won't have able to interfere and some clients end up with a fake blockchain, but I guess that can be remedied when the next block is received and it doesn't match.

I know I'm a bit vague, but that's the general idea - have a bittorrent client in the ethereum client that pushes and downloads a blockchain snapshot (e.g. each day / week) to a bittorrent DHT.

@fjl

This comment has been minimized.

Show comment
Hide comment
@fjl

fjl May 30, 2017

Contributor

One issue is that the blockchain changes all the time. New data is constantly created and would need to be pushed to the bittorrent network.

Contributor

fjl commented May 30, 2017

One issue is that the blockchain changes all the time. New data is constantly created and would need to be pushed to the bittorrent network.

@Glamdring

This comment has been minimized.

Show comment
Hide comment
@Glamdring

Glamdring May 30, 2017

Yup, that's what I meant - "pushes and downloads a blockchain snapshot (e.g. each day / week)". The thing is, data itself is not pushed to the bittorrent network - only the hash of the data, which is a way to locate the actual data on the clients' machines, where it resides anyway

Yup, that's what I meant - "pushes and downloads a blockchain snapshot (e.g. each day / week)". The thing is, data itself is not pushed to the bittorrent network - only the hash of the data, which is a way to locate the actual data on the clients' machines, where it resides anyway

@kdunn926

This comment has been minimized.

Show comment
Hide comment
@kdunn926

kdunn926 Jun 22, 2017

+1 to this idea - the blockchain sync process is quite slow at the moment and will likely create an artificial barrier to entry for other new users.

+1 to this idea - the blockchain sync process is quite slow at the moment and will likely create an artificial barrier to entry for other new users.

@JigarJoshi

This comment has been minimized.

Show comment
Hide comment
@JigarJoshi

JigarJoshi Jul 29, 2017

maps back to SNAPSHOT and release model of software development. I like the idea. decentralized repository of releases and a scrolling SNAPSHOT. 👍

maps back to SNAPSHOT and release model of software development. I like the idea. decentralized repository of releases and a scrolling SNAPSHOT. 👍

@zxsanny

This comment has been minimized.

Show comment
Hide comment
@zxsanny

zxsanny Jul 30, 2017

Can somebody share Ethereum geth\chaindata directory via
https://www.resilio.com/individuals/#
and share read-only key for that?

zxsanny commented Jul 30, 2017

Can somebody share Ethereum geth\chaindata directory via
https://www.resilio.com/individuals/#
and share read-only key for that?

@nicksavers

This comment has been minimized.

Show comment
Hide comment
@nicksavers

nicksavers Jul 30, 2017

Contributor

The slow sync process is probably not because of a slow download, but mostly because all blocks are fully processed to calculate and verify the state.

Blocks in a chain don't really change after a valid hash is found, so if a DHT-like network is actually faster, miners could upload the block to Swarm and clients could download the data from there after receiving the block-hashes.

Contributor

nicksavers commented Jul 30, 2017

The slow sync process is probably not because of a slow download, but mostly because all blocks are fully processed to calculate and verify the state.

Blocks in a chain don't really change after a valid hash is found, so if a DHT-like network is actually faster, miners could upload the block to Swarm and clients could download the data from there after receiving the block-hashes.

@mpcm

This comment has been minimized.

Show comment
Hide comment
@mpcm

mpcm Jan 11, 2018

So... I ended up here, because I assumed this (or something like it) was already being done.. either within or alongside geth. Then I recently setup a new system, and was surprised about the initial time to download (and cpu burn of that effort).

mpcm commented Jan 11, 2018

So... I ended up here, because I assumed this (or something like it) was already being done.. either within or alongside geth. Then I recently setup a new system, and was surprised about the initial time to download (and cpu burn of that effort).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment