Tracker announce causes chrome-extension to crash due to too many RTCPeerConnections construct

[reisti]

What version of WebTorrent?
0.98.24

What operating system and Node.js version?
Windows 10. Node.js: v 8.10.0

What browser and version? (if using WebTorrent in the browser)
Chrome 65.
What did you expect to happen?
First of all, thank you for a really awesome package! I am trying to create a chrome-extension to explore the possibility of a distributed P2P-based certificate transparency log and have got this to work. The approach is that every time the user visits a new tab, some certain information regarding the tab should be fetched as a torrent or from server if the tab is currently unseeded and then seeded by the user. This works really well for a number of tabs (around 6), more tabs then that results in an error.

Removing old torrents and on callback adding new and thus making sure i only have a maximum of around 6 torrents active at any given time does not fix the problem. I have viewed the sourcecode of Webtorrent and found that torrent.remove does indeed have the desired behaviour of also destroying all the peers.

What actually happened?
Not immeditially but always after some time (around 30-90 seconds) after exceeding 6 torrents, I get the following error. Notable is that if I open more than 6 tabs, and thus start to seed them, they are in fact seeded and other peers can come and get them. Until the trackers make an announce and the extension crashes.

Uncaught DOMException: Failed to construct 'RTCPeerConnection': Cannot create so many PeerConnections at new Peer (chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:19793:14) at WebSocketTracker._createPeer (chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:2121:14) at generateOffer (chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:2087:43) at WebSocketTracker._generateOffers (chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:2080:5) at WebSocketTracker.announce (chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:1769:10) at chrome-extension://opjhcaejljcceoepbhfbdegegedaeeap/bundle.js:1693:12

I have done some reasearch to this before posting this issue, and these seemded the most relevant:
Cannot create so many PeerConnections - This error was fixed when setting the RTCPeerConnection to null which is in fact done in webtorrent at Peer.destroy which is done on torrent.remove.
Too many RTC Peer Connections - Feross explained that many RTCPeerConnections are opened when the "RTC offers" generated by the extension are sent to the tracker servers.
RTCPeerConnection limit - As refered to here, the maximum number of RTCPeerConnections is 256 (without a source though). This is quickly hit when the RTC offers to the trackers are generated. On other note 256 does not seem to be true, more like 30.

For tracker config i am using:
global.WEBTORRENT_ANNOUNCE = createTorrent.announceList .map(function (arr) { return arr[0] }) .filter(function (url) { return url.indexOf('wss://') === 0 || url.indexOf('ws://') === 0 })
which would then refer to
module.exports.announceList = [ [ 'udp://tracker.leechers-paradise.org:6969' ], [ 'udp://tracker.coppersurfer.tk:6969' ], [ 'udp://tracker.opentrackr.org:1337' ], [ 'udp://explodie.org:6969' ], [ 'udp://tracker.empire-js.us:1337' ], [ 'wss://tracker.btorrent.xyz' ], [ 'wss://tracker.openwebtorrent.com' ], [ 'wss://tracker.fastcast.nz' ] ]¨
To create the client I use the following config (after following discussions in #288 about the DHT creating many RTCPeerConnection objects):
var client = new WebTorrent({ dht: false, tracker: { rtcConfig: null } })

So my questions are;

• Why are this error throwed? What is the limit?
• How can i controll the amount of RTCPeerConnections that are created?
• Can I catch this error? I have already tried, but for example client.on('error') does not work.
• How can I get RTCPeerConnections and destory them (can that even fix the problem?)
• Any thoughts on this at all...

[download13]

I've run into this problem as well, but it was with only one torrent left open for more than 15 minutes or so.

I found a question that might be relevant. Is this a garbage collection issue, or is the limit simply too low for webtorrent to work without bumping into it?

EDIT: It looks like the code for Peer properly nulls everything associated with the peer connection. I guess it's just creating too many Peer objects. There must be a way to limit this...

[arnWolff]

I encounter same issue.
As i can test it, passing 'numwant' option to each tracker announce url (e.g: wss://tracker.openwebtorrent.com?numwant=5) help to mitigate this bug but better i guess would be to fix it in webtorrent source, if it's not a bug relative to chrome?!

[feross]

Yeah, we ought to mitigate this directly in WebTorrent by reducing the peer connection timeout (so peers get destroyed faster) and lowering the default numwant.

A global resource limiter would also help, since right now the client has no way to know it's reaching the browser's limit. It'll just try to create peers whenever it wants to and will hit the limit. Related to #163.

[didlie]

Can you please specify the most recent previous release that didn't have this issue, and tell me how I can get an un-minified version of the build.

[chr15m]

I have some more data for this issue. As per original poster I'm also working with tabs connected to a single infoHash over long periods of time. I have numwant=5 set.

Symptoms I am seeing:

• In chrome:webrtc-internals up to 44 WebRTC connections getting created (see last screenshot).
• The chrome:webrtc-internals tab crashing(!).
• WebRTC connections remaining in that view long after the simple-peer debug messages say they have been destroyed.
• Failed to construct 'RTCPeerConnection': Cannot create so many PeerConnections

Suspect not all references are getting cleaned up immediately. Could be Chrome being slow to garbage collect those resources.

Some screenshots over about an hour of testing:

Start with numwant offers

Quite quickly exceed numwant outstanding offers

Many offers say they are closed but still appear

Chrome WebRTC internals tab crashes

Number of offers continues to fluctuate reaching 44

Will continue to investigate.

[chr15m]

This is a leak in Google Chrome which started to manifest in version 65.0.3300.0.

https://bugs.chromium.org/p/chromium/issues/detail?id=825576#c24

It's to do with the way in which Chrome garbage collects the old WebRTC candidates (or rather does not garbage collect them in a timely manner).

To replicate this:

• Tab 1: Open chrome://webrtc-internals/
• Tab 2: Go to https://chr15m.github.io/simple-peer-test/ (or create a simple-peer instance yourself with p = new Peer({ initiator: true}))
• Tab 1: Check to see a candidate has been created.
• Tab 2: In the Javascript console issue p.destroy().
• Tab 1: Note that candidate still persists.

The length of time the candidate hangs around is highly variable. In Firefox candidates are instantly cleared after destroy() is called.

From that ticket the code to replicate this issue is:

var i = 1;
function peer() {
  var peer = new RTCPeerConnection();
  setTimeout(() => {
    peer.close();
    peer=null;
  }, 10);
  console.log(i++);
}

setInterval(peer, 20);

This is similar to what is happening in bittorrent-tracker/lib/client/websocket-tracker.js but over a much shorter time frame. Hopefully the bug in Chrome gets fixed as it makes WebRTC with multiple connections quite fragile in that browser.

[didlie]

I ran a test on Google Chrome and the maximum number of RTCPeerConnections is 499 before this error appears. So there is a fundamental problem with the peer creation process in webtorrent. It does not create peers in an efficient way. And, as you pointed out, the run-time of any browser instance of webtorrent is limited , as the maximum number of allowed RTCPeerConnection is capped at 499. Peer connections must be created efficiently, or the browser will crash with the error you see. I also had to implement a pre-validation process for peer handshakes to avoid the 499 RTCPeerConnection cap in Chrome.... other browsers also have limits, and MS Edge simply does not allow the creation of a raw dataChannel.... perhapse because data networking is a product MS sells.... but thats beyond the point. Feel free to message me for help on this issue. I spent several months developing an algorithm to create fully connected peer data networks using WebRTC and I succeeded. I published my solution on Amazon . You can google: Serverless Javascript Networking Algorithm , by Isaac Jacobs, to see the published Javascript code solution.