Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Sharding introduction R&D compendium
Sharding+Casper/shasper with Ethereum 2.0 is the planned scaling approach for Ethereum; work is in progress on R&D.
Ethereum 1.0 can only process 7-15 transactions per second, the goal of sharding is to partition all network computational resources into shards, so that a node (a single computer as a peer connected to the network) doesn't have to process (download, compute, store, read) every transaction in the history of the blockchain, in order to make a new transaction (write and upload) or otherwise participate in securing and using Ethereum; rather a node can just participate in a single shard, or more if it so chooses. Multiple shards are handled separately by different subsets of securing participants, aka securitors (which include notaries, proposers, miners and validators). The primary goal is a massive scalability improvement, potentially exponential in phase 6 of the roadmap, although Vitalik questioned whether exponential sharding will even be tenable in an ethresear.ch thread. Quadratic sharding involves having shards at a depth of at most 1 from the main chain, that is, there are no shards within a shard, or a manager shard managing sub-shards; whereas exponential sharding has shards within shards, recursively.
Each one of the shards (currently set to 1024 in the latest shasper spec) will have as high a capacity (and likely more with phase 1) than the current existing Ethereum chain. The latest spec combines sharding with Casper FFG PoS (see e.g. the PoS FAQ and compendium for details) using a RANDAO beacon chain. Previously a contract was planned to exist on the main chain, however that has been scrapped, in order to make sharding implementation easier, as explained e.g. here in a Tweet storm by Vitalik.
A smart contract will exist on the main chain, called the sharding manager contract, which manages how data and transactions in shards are accepted as valid by the main chain, via notaries voting on the validity and data availability of collations (collections of data and transactions, analogous to blocks, but where they occur more frequently than blocks) in shards, and proposers proposing blobs (analogous to transactions but without execution in phase 1) which are collected into collations.
- Independent research
- Grant program
- Alternative scaling approaches to sharding
For information on sharding, refer to (sorted roughly from the most recent/important information to less recent):
- Sharding FAQs
- sharding roadmap
- a summary here;
- ethresear.ch sharding category (watch new posts);
- sharding protocol research tracker
- Sharding Fork Choice rule PoC and more info, 2018 May 1. Also see here for an issue tracking protocol specification updates.
- networking diagram on slides 82–87
- Sharding workshop notes
- Ethereum sharding workshop blog post
- HackMD note: Ethereum Research Sharding Compendium
- Wiki: ethresear.ch Sharding Compendium
Implementations under development include:
- sharding implementers call
- py-evm/Trinity: Gitter, Github: sharding, Github: Trinity. Trinity is the planned/under development EVM that will run a node and network.
- Prysm: Ethereum 2.0 client implemented in go. Gitter, website.
- Drops of Diamond: using the fast, safe and concurrent Rust programming language, implemented deprecated specifications, it's lead developer @jamesray1 has been working on rust-libp2p, particularly gossipsub, an extensible pubsub messaging protocol.
- Lighthouse: implemented the first version of the shasper spec, while work is still underway while waiting for the latest spec to finalize.
- Nimbus (doc): an implementation in Nim which compiles to C, and would be good for integrating with Hera, an eWASM implementation in C++. Developed by Status, they are looking to support mobile phones.
- PegaSys (ConsenSys): developing a full client that is targeted for enterprise users.
- deprecated: sharding utils: has the sharding manager contract and interfaces.
- Cambridge Army, @maxc on https://ethresear.ch, working on reducing the number of rounds of interactive verification e.g. with execution tracing, parallelism, or breaking up a transaction into smaller ones if a certain number of steps of verification have been reached.
Alternative scaling approaches to sharding
Enshrined-in-consensus: see https://ethresear.ch/t/sharding-phase-1-spec-retired/1407/18.