Update Networking Specification

[FlorianFranzen]

Requested specifically by implementors (in rough order of priority):

• More details about discovery process
• How exactly sync and block announce protocols work
• More details about other protocols if there are any (imonline?)
• More details about ephemeral streams
• More details about grandpa gossiping

Depends on #43 being resolved (and probably should also take a look at #20).

EDIT: Ready in PR #333

[FlorianFranzen]

@infinity0 I assigned this to you for now. What is your opinion on specifying these?

[infinity0]

My position is essentially unchanged since I was asked this several months ago.

As mentioned in #43, networking has various high-level and long-term changes planned. I have been working in conjunction with paritytech (the main implementers of polkadot) on designs and proposals for the various components. Most of these are done on a high (vague) level, the last remaining major component not yet started is XCMP networking. Most of these are not done on a low level, and it will be quite some time before we are in a position to write something like a IETF-style detailed RFC.

IMO it is not worth the effort to work on such a detailed spec at the current time, since its lifetime would be extremely limited and the cost benefit trade-off is not worth it. My projected estimate for when it would be appropriate to work on such a detailed spec, would be 3-6 months from now.

Any other implementers should understand that this situation I just described, is very normal for any type of research state-of-the-art project at this early stage of its lifetime, and not demand RFC-style specs at this stage. If they want to proceed with implementation, it is their responsibility to work more directly with paritytech, and ask more detailed questions than simply "please give me more details". (Compare the development of the early web, and of bitcoin, ethereum, etc etc etc). I can of course also help with that process and those discussions; what I cannot help with is an extremely generic request to "provide more details [about literally everything]", as the overall project is just not at that stage of its lifetime.

[FlorianFranzen]

Most of these are done on a high (vague) level, the last remaining major component not yet started is XCMP networking.

Where can I find the latest high level write-ups?

IMO it is not worth the effort to work on such a detailed spec at the current time, since its lifetime would be extremely limited and the cost benefit trade-off is not worth it. My projected estimate for when it would be appropriate to work on such a detailed spec, would be 3-6 months from now.

And that is ok. And I think this is also something we should clearly communicate in the spec (Appendix E) as well.

If they want to proceed with implementation, it is their responsibility to work more directly with paritytech, and ask more detailed questions than simply "please give me more details". (Compare the development of the early web, and of bitcoin, ethereum, etc etc etc). I can of course also help with that process and those discussions; what I cannot help with is an extremely generic request to "provide more details [about literally everything]", as the overall project is just not at that stage of its lifetime.

I understand your point. However, in this concrete case, we literally have a list of "requests for details" from an implementer that wants to start tracking upstream more closely. So the least we can do is share with them the current state, even if it is in form of code for now, and the high level idea behind it, to help them structure their code somewhat sensible.

[lamafab]

@lamafab will continue to expand the networking section, including updating the current information (certain specifications are outdated).

[drskalman]

@lamafab is currently working on speccing substreams and messages and handshakes.

[drskalman]

@drskalman is editing this and will push changes. Todos are inline.

[FlorianFranzen]

Related issues: #150, #131