Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

protocol-select/: Add Protocol Select specification #349

Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
83c79c6
protocol-select/: Add first draft
mxinden Jul 12, 2021
14d5f36
protocol-select/: Extend on uncoordinated TCP Simultaneous Open
mxinden Jul 13, 2021
cb68970
protocol-select/: Document protocol evolution
mxinden Jul 13, 2021
03e9e27
protocol-select/: Restructure message flow
mxinden Jul 13, 2021
780570b
protocol-select/: Move Protocol IDs to Extension section
mxinden Jul 13, 2021
c9f6814
protocol-select: Replace mention of muxer with generic protocol
mxinden Jul 13, 2021
aea6579
protocol-select: Extend FAQ section
mxinden Jul 13, 2021
a814240
protocol-select/README.md: Fix typo
mxinden Jul 19, 2021
1a9adc5
protocol-select/README.md: Fix typo
mxinden Jul 19, 2021
9bfe115
split the multiaddr change out of this spec
marten-seemann Jul 25, 2021
4ccea37
reword description of the challenges associated with upgrading multis…
marten-seemann Jul 25, 2021
7913830
add heading for Protocol Select connection establishment
marten-seemann Jul 26, 2021
fc07c19
Merge pull request #354 from libp2p/protocol-select-split-addr-change
marten-seemann Jul 27, 2021
f589bef
protocol-select/README.md: Use bytes for Protocol ID
mxinden Jul 27, 2021
843e760
protocols-select/: Describe remote accepting Protocol ID
mxinden Jul 27, 2021
b93ff63
protocol-select/: Wrap lines
mxinden Jul 27, 2021
6eb0023
protocol-select/: Add FAQ entry on Protocol IDs
mxinden Jul 27, 2021
24c9ea2
protocol-select/: Add FAQ entry on single message type
mxinden Jul 27, 2021
849b993
protocol-select/: Add FAQ entry on proto2 vs proto3
mxinden Jul 27, 2021
bb049b5
protocol-select/: Mention both NAT and firewalls
mxinden Jul 27, 2021
7886798
protocol-select/: Use dialer/listener instead of client/server
mxinden Jul 27, 2021
77470f0
protocol-select: Remove links in headings
mxinden Jul 27, 2021
b6b5d2e
connections/: Replace Multistream Select with Protocol Select
mxinden Jul 27, 2021
324ff28
protocol-select/: Remove reference to Offer and Use message
mxinden Jul 27, 2021
74ba8a3
protocol-select/: Document Protocol Name structure
mxinden Jul 27, 2021
3a0c2f3
Merge branch 'libp2p/master' into protocol-select
mxinden Jul 27, 2021
76d2d78
apply @yusefnapora's suggestions
marten-seemann Aug 25, 2021
6e947b8
protocol-select/README: Mark `version` field as `required`
mxinden Nov 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 21 additions & 58 deletions connections/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ and spec status.
- [Overview](#overview)
- [Definitions](#definitions)
- [Protocol Negotiation](#protocol-negotiation)
- [multistream-select](#multistream-select)
- [Protocol Select](#protocol-select)
- [Multistream Select](#multistream-select)
- [Upgrading Connections](#upgrading-connections)
- [Opening New Streams Over a Connection](#opening-new-streams-over-a-connection)
- [Practical Considerations](#practical-considerations)
Expand Down Expand Up @@ -113,8 +114,7 @@ Each protocol supported by a peer is identified using a unique string called a
**protocol id**. While any string can be used, the conventional format is a
path-like structure containing a short name and a version number, separated by
`/` characters. For example: `/mplex/1.0.0` identifies version 1.0.0 of the
[`mplex` stream multiplexing protocol][mplex]. multistream-select itself has a
protocol id of `/multistream/1.0.0`.
[`mplex` stream multiplexing protocol][mplex].

Including a version number in the protocol id simplifies the case where you want
to concurrently support multiple versions of a protocol, perhaps a stable version
Expand All @@ -125,10 +125,18 @@ receives the protocol id negotiated for each new stream, so it's possible to
register the same handler for multiple versions of a protocol and dynamically alter
functionality based on the version in use for a given stream.

### multistream-select
libp2p supports two protocol negotiation protocols, _Protocol Select_ and
_Multistream Select_, the former replacing the latter and the latter being
deprecated.

libp2p uses a protocol called multistream-select for protocol negotiation. Below
we cover the basics of multistream-select and its use in libp2p. For more
### Protocol Select

The _Protocol Select_ protocol as well as how it is embedded in libp2p is
described in the [_Protocol Select_ specification][protocol-select].

### Multistream Select

Below we cover the basics of multistream-select and its use in libp2p. For more
details, see [the multistream-select repository][mss].

Before engaging in the multistream-select negotiation process, it is assumed
Expand All @@ -152,6 +160,7 @@ hex):
The first byte is the varint-encoded length (`0x03`), followed by `na` (`0x6e 0x61`),
then the newline (`0x0a`).

Multistream-select itself has a protocol id of `/multistream/1.0.0`.

The basic multistream-select interaction flow looks like this:

Expand Down Expand Up @@ -182,6 +191,9 @@ traffic over the channel will adhere to the rules of the agreed-upon protocol.
If a peer receives a `"na"` response to a proposed protocol id, they can either
try again with a different protocol id or close the channel.

Note: In the case where both peers initially act as initiators, e.g. during NAT
hole punching, tie-breaking is done via the [multistream-select simultaneous
open protocol extension][simopen].

## Upgrading Connections

Expand All @@ -193,48 +205,17 @@ connections is called "upgrading" the connection.
Because there are many valid ways to provide the libp2p capabilities, the
connection upgrade process uses protocol negotiation to decide which specific
protocols to use for each capability. The protocol negotiation process uses
multistream-select as described in the [Protocol
_Protocol Select_ as described in the [Protocol
Negotiation](#protocol-negotiation) section.

When raw connections need both security and multiplexing, security is always
established first, and the negotiation for stream multiplexing takes place over
the encrypted channel.

Here's an example of the connection upgrade process:

![see conn-upgrade.plantuml for diagram source](conn-upgrade.svg)

First, the peers both send the multistream protocol id to establish that they'll
use multistream-select to negotiate protocols for the connection upgrade.

Next, the Initiator proposes the [TLS protocol][tls-libp2p] for encryption, but
the Responder rejects the proposal as they don't support TLS.

The Initiator then proposes the [Noise protocol][noise-spec], which is supported
by the Responder. The Listener echoes back the protocol id for Noise to indicate
agreement.

At this point the Noise protocol takes over, and the peers exchange the Noise
handshake to establish a secure channel. If the Noise handshake fails, the
connection establishment process aborts. If successful, the peers will use the
secured channel for all future communications, including the remainder of the
connection upgrade process.

Once security has been established, the peers negotiate which stream multiplexer
to use. The negotiation process works in the same manner as before, with the
dialing peer proposing a multiplexer by sending its protocol id, and the
listening peer responding by either echoing back the supported id or sending
`"na"` if the multiplexer is unsupported.

Once security and stream multiplexing are both established, the connection
upgrade process is complete, and both peers are able to use the resulting libp2p
connection to open new secure multiplexed streams.

Note: In the case where both peers initially act as initiators, e.g. during NAT
hole punching, tie-breaking is done via the [multistream-select simultaneous
open protocol extension][simopen].


## Opening New Streams Over a Connection

Once we've established a libp2p connection to another peer, new streams are
Expand All @@ -244,25 +225,12 @@ process](#upgrading-connections) if the transport lacks native multiplexing.
Either peer can open a new stream to the other over an existing connection.

When a new stream is opened, a protocol is negotiated using
`multistream-select`. The [protocol negotiation process](#protocol-negotiation)
_Protocol Select_. The [protocol negotiation process](#protocol-negotiation)
for new streams is very similar to the one used for upgrading connections.
However, while the security and stream multiplexing modules for connection
upgrades are typically libp2p framework components, the protocols negotiated for
new streams can be easily defined by libp2p applications.

Streams are routed to application-defined handler functions based on their
protocol id string. Incoming stream requests will propose a protocol id to use
for the stream using `multistream-select`, and the peer accepting the stream
request will determine if there are any registered handlers capable of handling
the protocol. If no handlers are found, the peer will respond to the proposal
with `"na"`.

When registering protocol handlers, it's possible to use a custom predicate or
"match function", which will receive incoming protocol ids and return a boolean
indicating whether the handler supports the protocol. This allows more flexible
behavior than exact literal matching, which is the default behavior if no match
function is provided.

## Practical Considerations

This section will go over a few aspects of connection establishment and state
Expand Down Expand Up @@ -354,12 +322,6 @@ See [hole punching][hole-punching] document.

## Future Work

A replacement for multistream-select is [being discussed][mss-2-pr] which
proposes solutions for several inefficiencies and shortcomings in the current
protocol negotiation and connection establishment process. The ideal outcome of
that discussion will require many changes to this document, once the new
multistream semantics are fully specified.

For connection management, there is currently a draft of a [connection manager
specification][connmgr-v2-spec] that may replace the current [connmgr
interface][connmgr-go-interface] in go-libp2p and may also form the basis of
Expand Down Expand Up @@ -409,3 +371,4 @@ updated to incorporate the changes.
[simopen]: ./simopen.md
[resource-manager-issue]: https://github.com/libp2p/go-libp2p/issues/635
[hole-punching]: ./hole-punching.md
[protocol-select]: ../protocol-select/README.md
Loading