Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Grandpa warp syncing #270

Closed
tomaka opened this issue Dec 4, 2020 · 4 comments
Closed

Implement Grandpa warp syncing #270

tomaka opened this issue Dec 4, 2020 · 4 comments

Comments

@tomaka
Copy link
Contributor

tomaka commented Dec 4, 2020

The sending side of Grandpa warp sync is paritytech/substrate#1208
This issue consists in implementing the "receiving" side, in other words sending a request, downloading the data, and syncing the chain.

There are two things to do:

  1. Add the system that analyzes the Grandpa warp sync payload and advances the chain.
  2. Add the networking code that lets you download that Grandpa warp sync payload.

Advancing the chain

The finalized state of the chain is represented in the code by a chain::chain_information::ChainInformation.
As a high level overview, advancing the chain would consist in updating this ChainInformation. In other words, some system that takes as input a ChainInformation and a warp sync payload, and outputs a new ChainInformation.

For logging purposes, it is important that we don't verify the entire warp sync payload at once, but step by step (one header and justification at a time).

As such, the API should look like:

pub struct Config {
    pub start: ChainInformation,
    pub warp_sync_payload: Vec<WarpSyncEntry>,
}

pub struct WarpSyncEntry {
    pub scale_encoded_header: Vec<u8>,  // please stick to Vec<u8>, no stronger typing
    pub scale_encoded_justification: Vec<u8>,  // same
}

pub enum Verification {
    Done(ChainInformation),
    InProgress(Verifier),
}

pub struct Verifier {
}

impl Verifier {
   pub fn new(config: Config) -> Self { ... }

   pub fn next(self) -> Verification { ... }

   pub fn state(&self) -> ChainInformationRef { ... }    // `Ref` as it's a reference to a ChainInformation
}

This can be put in a new module chain::warp_sync.

As for the implementation, it could use the NonFinalizedTree type found in blocks_tree.rs.
Unfortunately, the NonFinalizedTree requires you to pass all headers, and doesn't support skipping headers yet. It's unclear to me how to implement this, because one of the important step when advancing the finalized block is scanning all the headers in between the previous and new finalized blocks in order to find the new values to put in the updated ChainInformation.

It is unclear how to implement .

Adding the networking code

The code that decodes the payload should be put in protocol.rs. Similar existing code can already be found there for block requests and storage proof requests.

In the same vain, a function to send out a warp sync request should be added to src/network/service.rs and full_node/src/network_service.rs, similar to the existing blocks_request and storage_proof_request.

@tomaka
Copy link
Contributor Author

tomaka commented Jan 6, 2021

cc #294

@tomaka
Copy link
Contributor Author

tomaka commented Jan 6, 2021

After paritytech/substrate#7789 and paritytech/substrate#7829, which, assuming doesn't take ages to get merged, should be available in Polkadot 0.7.28, maybe at the same time as the GrandPa warp sync server side.

It should then be possible to grab the Babe epoch information from the single latest finalized block, without having to go through extra block requests to walk up the chain.

@tomaka
Copy link
Contributor Author

tomaka commented Jan 22, 2021

should be available in Polkadot 0.7.28

Will indeed be the case.

@tomaka
Copy link
Contributor Author

tomaka commented Feb 1, 2021

#430

@tomaka tomaka closed this as completed Feb 1, 2021
stiiifff pushed a commit to stiiifff/smoldot-old that referenced this issue Mar 20, 2023
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.9.0 to 0.10.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](jeromefroe/lru-rs@0.9.0...0.10.0)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant