PEX over WebRTC in browser

[miketamis]

Does PEX work over WebRTC, ie once the tracker bootstraps you into a swarm is the tracker still needed to get more peers.

[feross]

Currently the tracker is required for all WebRTC signaling (aka peer introductions).

PEX for WebRTC would be great. If someone would like to look into how this could be designed in a way that respects the existing design of PEX as much as possible, I'd be curious to hear it.

[KayleePop]

EDIT: See below for a better simplified version.

We could use other members of the swarm to forward signalling data.

Added list

Each peer in the added list includes the necessary hops over WRTC connections to reach them. Instead of a list of IPs, it's a list of paths using peer_ids: "<this client's id>/<directly connected peer's id>/<intermediary peer id>/<destination peer id>".

Peers that support the extension are added to the list when successfully connected to. "<this client's id>/<newly connected peer>"

If a peer doesn't support the extension, then they couldn't be connected to because they won't respond to forwarded signalling data.

Rules for adding a path to the list

(for adding paths from PEX messages too)

In order:

1. If your id isn't at the beginning of the path, then add it there.
2. If the same peer appears twice in a path, the unnecessary loop is removed. (usually a loop back to you after adding your ID)
3. If a path leading to the same final peer is already in the added list, then only keep the shortest of the two paths.

Dropped list

When a direct connection to a peer is broken, it is added to the dropped list like this: "<your id>/<previously connected peer>"

The reverse of the ids is the same connection, and that should be considered when making sure to not add duplicate dropped connections.

Every path in the added list that uses a dropped connection for any hop (including in the reverse direction) is removed.

PEX message

Added paths and dropped connections are shared without changes.

Dropped connections received via a PEX message are added without any changes.

See above rules for adding the paths from PEX messages to the added list.

Signal data forwarding

Now, instead of sending WRTC signalling data through the tracker, it can be sent through other peers if a path to it exists in the added list.

A new message type is sent to the directly connected peer in the path. The message includes the path string from the added list and the signalling data for establishing a WRTC connection. Intermediaries simply find their ID in the path and then forward the unedited message to the next peer. The final peer sends the answer back the same way by either using their stored path for that peer, or by reversing the order of the path in the message.

If an intermediary doesn't have the required connection to forward the message, then it sends back a dropped connection error message using the same path in reverse. This error contains the path string from the message and the dropped connection string "<peer sending the error>/<peer not connected to>". The peers in the path that are after the dropped connection can be truncated.

The dropped connection is added to the dropped lists of the original sender and the intermediaries used to forward the error. This allows dropped connections to propogate more quickly throughout the swarm.

The path in the forwarded message can also be added by intermediaries after forwarding by removing all peers before their id.

 

Possible improvements

1. More paths could be derived from the same information if a graph was used for local data instead of the path lists. Every connection in each path would be added as a bi-directional edge. Then every node could be converted into a path list by finding a route to it.
2. A nested list might be better than a single list of path strings for PEX messages to reduce redundancy.
3. Peer_ids are long, and I don't think the full length is necessary to prevent collisions within each swarm. It could be truncated to reduce bandwidth. (From the right to not include client info)

[feross]

What if we simplified to just include one hop? In other words, a peer sends a list of all peer IDs they're connected to during the extended handshake. Then the receiving peer can look through the list for peer IDs they're not connected to and ask for introductions (i.e. WebRTC signaling).

And just like the original PEX, we can notify peers whenever we learn of a new peer, so they always have up-to-date info about potential peers they can connect to, if they're interested.

Why is the multi-hop approach necessary?

[feross]

cc @jhiesey for feedback

[KayleePop]

Yeah it's not necessary you're right.

I thought that ut_pex exchanged peers received from other PEX messages without connecting to them, but it doesn't.

 

So the only changes would be:

• Use peer_ids instead of IP addresses
• Only include peers that support the extension in PEX messages
• new message type for forwarding signalling data
  • Includes
    • destination peer's id
    • sender's id
    • signalling data
  • Remove destination id before forwarding. If that field isn't included, then the signal is for you.
  • Use the same message type to send the WRTC response. The old sender id is used as the new destination id

[feross]

@KayleePop Yep, that sounds perfect to me!