Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Extract availability store networking to the availability-store crate #794

Closed
rphmeier opened this issue Jan 23, 2020 · 3 comments
Closed
Labels
I8-refactor Code needs refactoring.

Comments

@rphmeier
Copy link
Contributor

rphmeier commented Jan 23, 2020

The goal would be to have fully self-contained logic for availability networking within its crate. The refactorings in #788 may help this along, by making it easier to expose futures-based APIs for finding and communicating with validators.

This may use the network-gossip crate for now, but would in the future adapt to use point-to-point communication.

Other refactorings that could go in at the same time:

  • removing index from the erasure_coding_topic.
  • replacing unbounded channels with bounded ones.
@rphmeier rphmeier added the I8-refactor Code needs refactoring. label Jan 23, 2020
@rphmeier rphmeier added this to Not Started in Parachains Phase 1 Jan 23, 2020
@burdges
Copy link
Contributor

burdges commented Jan 26, 2020

We eventually want a transfer scheme with these properties:

  • Identifies pieces' current availability using some availability/tracker trait.
  • Routes piece retrieval across multiple hops using a some routing trait the caller implements.
  • Accommodates arbitrary data, not only our piece encoded PoV blocks, perhaps even with semantic dividing lines, like file boundaries in an archive format, but..
  • Also recognizes our Merkle proofs that authenticate pieces from their Merkle root, although conceivably this could be layered above somehow.

I think rustorrent already does the first three, but uses bittorrent's merklee tree for the last of course. We're almost surely better off implementing our own design on top of libp2p however, not rustorrent, because

  • we should reuse the same connections used the routing trait here for other purposes, like gossip,
  • we want QUIC's TCP style flow control, not the "give everybody else priority" flow control of Bittorrent's uTP, and
  • we should avoid solving other networking problems like crypto twice, etc.

I'll point out the BEPs list though anyways.

cc @infinity0

@expenses
Copy link
Contributor

I think we should wait for #860 to get started on this.

@rphmeier
Copy link
Contributor Author

rphmeier commented Feb 27, 2020

I had a call this morning with @AlistairStewart and @burdges about certain changes we might want to make to the availability networking - they will file an issue soon with summary. That should also shed some context on what we need.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
I8-refactor Code needs refactoring.
Projects
No open projects
Development

No branches or pull requests

3 participants