Sharding introduction R&D compendium

James Ray edited this page Oct 17, 2018 · 53 revisions

Documentation chat Sharding

Sharding+Casper/shasper with Ethereum 2.0 is the planned scaling approach for Ethereum; work is in progress on R&D.

Ethereum 1.0 can only process 7-15 transactions per second, the goal of sharding is to partition all network computational resources into shards, so that a node (a single computer as a peer connected to the network) doesn't have to process (download, compute, store, read) every transaction in the history of the blockchain, in order to make a new transaction (write and upload) or otherwise participate in securing and using Ethereum; rather a node can just participate in a single shard, or more if it so chooses. Multiple shards are handled separately by different subsets of securing participants, aka securitors (which include notaries, proposers, miners and validators)[1]. The primary goal is a massive scalability improvement, potentially exponential in phase 6 of the roadmap, although Vitalik questioned whether exponential sharding will even be tenable in an ethresear.ch thread. Quadratic sharding involves having shards at a depth of at most 1 from the main chain, that is, there are no shards within a shard, or a manager shard managing sub-shards; whereas exponential sharding has shards within shards, recursively.

Each one of the shards (currently set to 1024 in the latest shasper spec) will have as high a capacity (and likely more with phase 1) than the current existing Ethereum chain. The latest spec combines sharding with Casper FFG PoS (see e.g. the PoS FAQ and compendium for details) using a RANDAO beacon chain. Previously a contract was planned to exist on the main chain, however that has been scrapped, in order to make sharding implementation easier, as explained e.g. here in a Tweet storm by Vitalik. A smart contract will exist on the main chain, called the sharding manager contract, which manages how data and transactions in shards are accepted as valid by the main chain, via notaries voting on the validity and data availability of collations (collections of data and transactions, analogous to blocks, but where they occur more frequently than blocks) in shards, and proposers proposing blobs (analogous to transactions but without execution in phase 1) which are collected into collations.

Contents

Information

For information on sharding, refer to (sorted roughly from the most recent/important information to less recent):

Implementations

Implementations under development include:

  • sharding implementers call
  • py-evm/Trinity: Gitter, Github: sharding, Github: Trinity. Trinity is the planned/under development EVM that will run a node and network.
  • Prysm: Ethereum 2.0 client implemented in go. Gitter, website.
  • Drops of Diamond: using the fast, safe and concurrent Rust programming language, implemented deprecated specifications, it's lead developer @jamesray1 has been working on rust-libp2p, particularly gossipsub, an extensible pubsub messaging protocol.
  • Lighthouse: implemented the first version of the shasper spec, while work is still underway while waiting for the latest spec to finalize.
  • Nimbus (doc): an implementation in Nim which compiles to C, and would be good for integrating with Hera, an eWASM implementation in C++. Developed by Status, they are looking to support mobile phones.
  • PegaSys (ConsenSys): developing a full client that is targeted for enterprise users.
  • deprecated: sharding utils: has the sharding manager contract and interfaces.

Independent research

  • Cambridge Army, @maxc on https://ethresear.ch, working on reducing the number of rounds of interactive verification e.g. with execution tracing, parallelism, or breaking up a transaction into smaller ones if a certain number of steps of verification have been reached.

Grant program

https://blog.ethereum.org/2018/01/02/ethereum-scalability-research-development-subsidy-programs/

Also see https://github.com/Drops-of-Diamond/diamond_drops/issues/18.

Alternative scaling approaches to sharding

Moved to https://github.com/ethereum/wiki/wiki/Alternative-blockchains,-randomness,-economics,-and-other-research-topics.

For precursor research scaling ideas, see para 2. here (see the links to hypercubes and Chain Fibers—a precursor to sharding, as well as hub-and-spoke chains).

Scaling proposals

Glossary

Enshrined-in-consensus: see https://ethresear.ch/t/sharding-phase-1-spec-retired/1407/18.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.