Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared testnet roadmap #258

Open
zah opened this issue Apr 30, 2019 · 7 comments

Comments

@zah
Copy link
Member

commented Apr 30, 2019

To arrive at a working shared testnet, we'll need to be able to monitor a validator deposit contract running in a ETH1 testnet, to conform to all ETH2 test vectors published in the next 6 months and to have a fully-compatible networking stack.

Our development environment will be enhanced to allow everyone to run a local simulation of a complete shared testnet (involving Nimbus, ETH1 testnet and other clients) and we'll be maintaining our own staging public testnet running on our existing infrastructure.

Tasks:

ETH1 integration (Yuriy, Bruno)

  • Create scripts for starting a local ETH1 test chain (using Genache or Embark simulator)
  • Create a CLI for creating deposits
  • Finish nim-web3
  • Implement the validator contract monitoring. Detect the genesis and post-genesis conditions
  • Deposit merkle proof (jangko, Mamy)

Spec updates (Mamy, Dustin)

  • Create a test suite using the official ETH2 test vectors and follow the spec updates
  • Implement the new SSZ spec (SOS)
  • Optimized state transitions (once test vectors are present)
  • Merkle performance (magic macro cache/diff/whatever solution)
  • support 100k validators
  • 1 beacon node + 1 validator on a Pi3b+
  • Implement slashing condition detection
  • Optimized LMD ghost (bitwise, incremental)

Simulation (Zah, Mamy, Jacek)

  • Enhance nim-vagrant to support compiling and debugging Prysm and Lighthouse.
    See status-im/nimbus#172 for more details.
  • Create scripts for launching multi-client testnets and integration tests involving multiple clients.
  • Deterministic startup / private key generation

Validators UX

  • Implement a minimal validator WebUI
  • Implement the slashing conditions and their UX
  • Documentation for end users
  • Introduce the validator/beacon-node split

ETH2 wire protocol (Yurii, Eugene)

  • LibP2P back-ends for the high-level p2pProtocols
  • Block Request manager
  • Syncing blocks
  • Networking spec compatibility
  • Block validation before propagation
  • peer scores (feed back useful peer information)
  • Keep making progress on a native Lib2P2 implementation
  • Discovery v5 (stretch)

Light clients

  • Generate and validate Merkle proofs for arbitrary SSZ interior values
  • Generate and validate Merkle proofs derived from executed functions
  • Implement a PoC light client protocol
  • Light client sync algorithm

Monitoring enhancements (Stefan, Zah)

  • Start tracking key performance metrics over time in CI
  • Implement a library for metrics
  • Implement a JSON-RPC interface (investigate during interop)
  • Collaborate with Embark on the creation of beacon chain block explorer
  • Testnet health page
  • Alerts

Phase1 preparations

  • Broadcast shard attestations and blocks generated by persistent committes (use separate topic per shard).
  • Implement per-shard fork choice
  • Implement the custody game
@Swader

This comment has been minimized.

Copy link
Contributor

commented May 1, 2019

Applying for:

  • Validators UX: minimal WebUI
  • Validators UX: docs
  • Eth1 testing documenting everything
  • Monitoring enhancements (all)
@tersec

This comment has been minimized.

Copy link
Contributor

commented May 1, 2019

Interested in:

  • Optimized state transitions (once test vectors are present)
  • Generate and validate Merkle proofs for arbitrary SSZ interior values
  • Generate and validate Merkle proofs derived from executed functions

@mratsim mratsim pinned this issue May 1, 2019

@Swader

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

  • Create a CLI for creating deposits
  • Implement the validator contract monitoring. Detect the genesis and post-genesis conditions

^ this is a little weird to me, why do we need this? If we finish the rest, this is implied and easy to make.

I would remove those. I would also remove:

  • Create scripts for starting a local ETH1 test chain (using Genache or Embark simulator)

The above seems unnecessary, especially before we have web3 finished or a spec-compatible RPC and an optimized VM that can run all Geth blocks.

I would add the following goal for Eth1:

  • achieve size parity with Geth (max 200 GB for a full sync). This would require us to also have pruning since Nimbus currently does an archive sync IIRC. This does not mean Nimbus should be production-ready, and we should not advertise it as such because we have no support bandwidth, but that we need to work on storing the data in a way that'll lend itself nicely to Eth 2.0

IRT - [ ] Implement the slashing conditions and their UX - let's define RPC interfaces and/or event/log messages for these conditions and stick by them. We need a way for clients to reliably detect that they're being slashed and for what reason - I want this to be very, very clear in the UI. Is anyone familiar with any discussions underway about standardizing Validator status reports?

@zah

This comment has been minimized.

Copy link
Member Author

commented May 9, 2019

Q: Why do we need CLI tools for creating deposits?

These will be very rudimentary tools used by all testing and simulation scripts. This also covers the next question - we need local simulations during development in order to test our code :)

@mratsim

This comment has been minimized.

Copy link
Member

commented May 9, 2019

Interested in:

  • Create a test suite using the official ETH2 test vectors and follow the spec updates
  • Optimized LMD ghost (bitwise, incremental)
  • Optimized state transitions (once test vectors are present)
  • Generate and validate Merkle proofs for arbitrary SSZ interior values
  • Generate and validate Merkle proofs derived from executed functions

And what is not listed:

  • optimizing Eth1
  • the VM phase 2
  • data viz on the metrics collected
@mratsim

This comment has been minimized.

Copy link
Member

commented May 9, 2019

I remembered what I forgot to say yesterday.

We need to be able to load the chain constants from a configuration file at compile-time (using staticRead/gorge I guess).

Presets are stored here: https://github.com/ethereum/eth2.0-specs/tree/d1c96c1e0d3b97ac6b436cbaa070e4a39f6b5876/configs/constant_presets

@Swader

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

bar.changeHeight()

We must now aim to have Ethereum 1.0 in 50GB or less with fast sync: https://www.reddit.com/r/ethereum/comments/bmlp2u/nethermind_099_45gb_mainnet_fast_sync_in_5_hours/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.