Publish a BEP explaining how to add support for WebTorrent

[yciabaud]

This PR proposes a BEP document as discussed in #168

I would like to have some feedback on the content and on some changes I made in the messages.
I changed some properties to have a cleaner protocol, if we keep it, we will have to update bittorrent-tracker to fit with this document.

You can view the html output on github: https://github.com/yciabaud/webtorrent/blob/486a94d2b651e7aed86ba27b82e679f5d1d1e700/bep_webrtc.rst

@feross @DiegoRBaquero @bbarrows

[yciabaud]

I changed the scrape response format from "infoHash" to "info_hash" to be consistent with the other messages.

[wI2L]

Actually I "think" (still not sure) that the response is created here : https://github.com/feross/bittorrent-tracker/blob/master/server.js#L698-L701, and that the "infoHash" field is used as a key in the forEachloop when creating a new entry in files list.

To me, it seems that either for a single or multi scrape request the response will contains a field list where the keys are the info_hash and the value an object with the stats of the related swarm.

[yciabaud]

Same thing for the multiscrape...

[yciabaud]

Here I changed from "failure reason" to "error" to be cleaner.

[alxhotel]

Looking at the UDP tracker BEP, I've found it uses a type of action called error whenever an error is return. And then a message attribute for the error message.

It could make sense adding it (at least the action attribute), just to be consistent in the response.

[yciabaud]

I agree with you, it is definitely more consistent with the other messages. Pushing it right now!

[yciabaud]

I changed from "answer" to "sdp" as this property can be either an offer or a response.

[wI2L]

I am thinking that a message like this one is not really related to Bittorrent annoucing, and that maybe we could use two new action events offer and answer. It would simplify server side parsing of messages and ensure that we know exactly how to deal with sdp.

EDIT: It seems to be either offer or answer in the payloads returned by tracker.openwebtorrent.com, and this field is an object containing the fields type and sdp. type seems to be consistent with the name of the field.

[lmatteis]

I'm a bit confused. So trackers supporting this would be able to track IP addresses of "normal" clients and also WebRTC contacts, right? That's cool but if WebRTC tabs can't connect to current BitTorrent client nodes (because most torrent clients don't support WebRTC), what's the point?

[yciabaud]

The point of this BEP is to document the process of signalling webrtc peers on a tracker to connect each other.

Aside from the tracker implementation, the clients supporting this would become hybrid clients and could use webrtc to connect to browser clients.

This would help a lot to make bittorrent content available in webtorrent network.

Do you understand now? Do you have an idea to make it clearer for a reader to understand what we are aiming?

[yciabaud]

The point is to make clients support webrtc by giving them a standard documentation ;)

[DiegoRBaquero]

Just read it all. It's very technical to the point of a BEP. Is there any place where I can see the guideline for a BEP?

Looking good @yciabaud

[yciabaud]

Thank you for reviewing @DiegoRBaquero.
This page details the BEP process: http://www.bittorrent.org/beps/bep_0001.html, some other guidelines can be found in http://www.bittorrent.org/beps/bep_0002.html

[austlane]

@yciabaud Thanks so much for writing this up. Looks detailed and well done!
I've been following this project for a number of years, hoping feross would do this sooner rather than later.

[yciabaud]

Thank you very much @austlane, lets hope we will push that to bittorrent soon!

[yciabaud]

I changed the error response message for it to look like the other messages like in other beps.

[wI2L]

Tbh, I think that both failure reason and an extra error field are necesseray. The first one wpuld stay for legacy reasons and would be used only for Bittorrent protocol related failures (either announce or scrape) while another extra field (or object, with code/message ?) could describe a signalling error.

[wI2L]

@yciabaud Just to clarify, is this BEP based on the actual implementation ? I couldn't find the error message you describe in the source code of the bittorrent-tracker project, instead it send back a failure reason key.

[yciabaud]

@wI2L I made some changes that I have commented inline in this PR. You are right, ATM the errors are labelled with failure reason.

I changed a bit some of the payloads too, all changes are inline commented.

[wI2L]

@yciabaud Ok. I have added comments on some.
While reading the source of bittorrent-tracker I have noticed the use of common.hexToBinary (for info_hash) and I am not sure of how to interpret that. Like I have mentionned it in our mail exchanges, the "type" of each field could be described.
Also, the websocket protocol describes binary and text messages. it may be necessary to define what kind of messages are supposed to be sent and read ?

EDIT: Messages are text, but encoding of info_hash and peer_id fields is binary. What is the reason behind that? This should be mentionned in the BEP.

[wI2L]

Analyzing a bit the payloads from wss://tracker.openwebtorrent.com/, this field is not present for announce done at regular intervals. However, for consistency with the BT protocol it should be mentionned that a missing event, or an event that is represented by an empty string is one of those made at regular intervals.

[wI2L]

answer is an object with the fields type and sdp.

[yciabaud]

Thanks @wI2L for these reviews, I agree that the types of the fields must be added to the BEP.
I will try to find a moment to work again on this.

[wI2L]

np @yciabaud
Also, I was wondering if the BEP should indicates how a BitTorrent tracker answer to a Webclient before the connection is actually upgraded to the Websocket protocol, in case of errors.
A standard tracker send back a bencoded dict with a failure reason key, with status code 200.

[Miserlou]

I STRONGLY recommend that we formally drop support for insecure websockets. It's almost 2017, there is no reason to introduce a potential security vulnerability or vector for surveillance. Especially with Let's Encrypt available for free, there is simply no excuse for running an insecure public service. WebRTC itself already disallows insecure connections, we should incorporate that momentum into this BEP.

I suggest we change this explicitly disallow 'ws' and only allow 'wss' here.

[wI2L]

According to https://github.com/feross/bittorrent-tracker/blob/master/server.js#L691-L695 and https://github.com/feross/bittorrent-tracker/blob/master/server.js#L698-L701, there is no shuch field announce.

[wI2L]

According to https://github.com/feross/bittorrent-tracker/blob/master/server.js#L677-L680, there is no such field announce.

[DiegoRBaquero]

Is there anything missing so this can be sent?

[wI2L]

@DiegoRBaquero Depends, would you like with @feross and @yciabaud to address my proposals I have made in the PR here: yciabaud#1
There is two ideas, publish a document that describes the actual state of the protocol, which could be done by finishing/polishing the original draft of @yciabaud or discuss furthermore about the changes I intoduced in favor of a more comprehensive protocol.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[rom1504]

Why is this blocked ? It seems important for webtorrent and isn't most of the work already done ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[feross]

Unlocking and re-opening this.