Server requirements

[NovoeVideo]

Hi guys!

Please calrify, what are server requirements for installing WebTorrent on my web site for streaming videos?

First of all, BANDWIDTH! Does the traffic from torrent file are coming through the hosting server?

Thank you.

[DiegoRBaquero]

There's two roles a server can play, tracker and seeder. If you are tracking your site's users to connect them, bandwidth is low, but memory requirement grows with your users, start with at least 512MB server and expect to double per 500 users or so.

In terms of (web)seeding, you would need a lot of bandwidth, so try running in an unlimited bw server.

[NovoeVideo]

Thanks Diego!

So if I add Magnet link (or upload file "filename.torrent") with video (NOT THE VIDEO ITSELF) through my console to web site, and then visitors just open the page and watching this video, so am I tracker, right?

Is this site: https://www.bitchute.com/video/EJoFbcWKeP0/ working as I described above???

[DiegoRBaquero]

No, Bitchute uses default trackers, maybe their own too, haven't checked.

If you seed a torrent you are a peer for that torrent and trackers tracker you in that torrent so other peers can connect to you.

[SilentBot1]

Hey @NovoeVideo,

Using Bitchute as an example, it uses the default trackers of btorrent.xyz, fastcast.nz and openwebtorrent.com.
The content for the torrent is loaded from three different webseeds located at seed1.bitchute.com, seed2.bitchute.com and seed3.bitchute.com respectively which are running NGINX as the webserver.
The torrent file is loaded from bitchute.com/torrent/[videoNumber]/[videoHash].webtorrent.

To implement web torrent on a website in the same way that Bitchute has, you would only need a webserver which has both the torrents content and the .torrent file. Since Bitchute uses the public websocket trackers, you wouldn't need to host your own but this I personally would so you can ensure availability if the other trackers go down.

You can assume due to the short lifespan of the default webtorrent implementation the majority of the bandwidth of the torrent will come from these webseeds, so the bandwidth needed will depend on your torrent and the number of visitors.

Hope this has helped!

[DiegoRBaquero]

@SilentBot1 thanks for looking into their implementation and the thorough explanation!

[Er-rchydy]

@SilentBot1 you explained how Bitchute work thanks for that, but i have a question, you said that "The content for the torrent is loaded from three different webseeds located at seed1.bitchute.com, seed2.bitchute.com and seed3.bitchute.com respectively which are running NGINX as the webserver. "
how does they do that ? i mean does those 3 webseeds seed the actual file for example video1.mp4 or they running webtorrent-hybrid and they seed the torrent file, in other word does Bitchute use webtorrent in both client side and server side or they use it only in client side to share the client's bandwidth, and if there are only one or couple clients they serve the actual real file from their servers. i hope you understand my question

[SilentBot1]

@Er-rchydy Just to state I'm not the developer nor maintainer of Bitchute. What I'm saying could be completely incorrect.

which are running NGINX as the webserver

To correct my previous statement, the actual web seeds are running express but they are also passing through a NGINX server, this can be seen by both the Server: nginx/1.6.2 and X-Powered-By: Express response headers, likely using NGINX as a loadbalancer or just using it for SSL termination.

i mean does those 3 webseeds seed the actual file

These peers are webseeds which operate from a express application; these express applications could be serving the range requests from fs.createReadStream() on a local file or from a webtorrent client using any chunk store via client.torrents[X].files[Y].createReadStream(). Using the latter option would keep the torrent in the webtorrent client and keep seeding it on the bittorrent network also making it easier for the initial distribution between all the webseeds.

When I downloaded an example file I only received webseed peers, combined with this issue with webtorrent-hybrid I doubt they are using webtorrent-hybrid on the back end as I didn't receive any non-webseed peers.

Without further clarification from the actual developer there is no clear way to know how they have achieved this result; though I know it's possible to use a webtorrent client and respond to the http range requests using client.torrents[X].files[Y].createReadStream() which will then prioritise the requested ranges from the bittorrent network as I've created a similar implementation.

Please correct me if anything you spot is incorrect,
Hope this helps!

[Er-rchydy]

@ @SilentBot1 thanks for the detailed response.
i I doubt they are using webtorrent-hybrid too, i've tried it on many OS but i got the same issue.
I saw in some response from this guy from PeerTube that they faced the same problems when they was using webtorrent-hybrid before, now they are using webseed.
Do you have any information about how webseeds and peers are working ? like if there are one seeder for file.mp4 for example, when does webtorrent try to download from the webseeds instead of peers ? if there are no peers then it makes sense to download from webseeds, but i don't know how webtorrent act when there are peers and webseeds.
thanks

[SilentBot1]

@Er-rchydy it's no problem.

I have an working example of a torrent with both a webrtc peer also with a webseed peer; the client seems to download the files roughly 50/50 between the different peers, indifferent to the type of seed. The testing environment was on a single host so latency and bandwidth wasn't a factor, I would be interested in seeing how varying latency or bandwidth between both the webseeds, webrtc peers and the client alter this ratio.

I understand what you're saying and It would be possible to add additional custom logic to only download from a webseed if the availability from webrtc peers is < 1 to promote inter peer downloading, using the webseed as a fallback, but I think this is a issue as webtorrents are so short lived and it would require scraping the trackers (or DHT) before downloading to ensure availability, causing additional delay on torrent addition, this something which I personally think, if implemented, it should be implemented at the library level and not the application level.

All the best.

[feross]

Feel free to continue this discussion. I'm just going to close this issue since it's not a bug with WebTorrent.

[Er-rchydy]

@SilentBot1 thanks that what i was looking for. so the client doesn't distinguish between the type of seed.
and i agree with you, it will be nice if some features about that was added to core of webtorrent. like to define the max number of peers who are allowed to connect the webseed.
Most people are using webtorrent to reduce the server's bandwidth. and you can't get that when you want to avoid dead-share and at the same time control your server's bandwidth.

[SilentBot1]

@Er-rchydy an example of removing webseeds if the swarm has over an a certain amount of completed peers reported by trackers could look something like this.

client.on('torrent', torrent=>{
	var trackerClient = torrent.discovery.tracker
	var urlList = torrent.urlList;
	torrent.urlList = []
	for (var pid in torrent._peers){
		if(torrent._peers[pid].type === "webSeed"){
			urlList.push(pid)
			torrent._peers[pid].destroy()
		}
	}
	var complete = []
	trackerClient.on('scrape', data=>{
		complete[data.announce] = data.complete
		var averageComplete = Object.keys(complete).reduce((a,b)=>{return a+complete[b]}, 0)/Object.keys(complete).length
		//< 1 determines how few completed peers you want before re-adding webseeds to avoid dead-share.
		if(averageComplete < 1){
			urlList.map(url=>{
				torrent.addWebSeed(url)
			})
		}
	})
	trackerClient.scrape()
})

This is similar to what you were asking as it would help promote more inter peer sharing while reducing the load on the webseeds. The downside to this approach is that you can't fully determine the health of the swarm by completed peers alone, the swarm upload speed may still be extremely slow.

All the best.

[Er-rchydy]

@SilentBot1 thanks for the example it was really helpful, i have a question , does doing this slowdown loading files, i mean does it has impact on latency somehow, cuz i'm trying to use it in a streaming live through webtorrent (1 to 5 seconds latency is still good for me)

[SilentBot1]

@Er-rchydy since in the browser, webtorrent uses websockets for communication with the trackers; the time it takes to complete a scrape request depends on your ping to the closest tracker, though the whole process of scraping only took 29ms for me to get a response from the first tracker which is enough information (to begin with) to re-add the webseed if there aren't enough peers.

(Green is request, red is response)

For livestreaming using webtorrent, assuming you're chunking up the stream into segments and seeding them from the browser, I'd imagine the biggest factor which will add latency is wrtc peer discovery or the current issue where client.seed will verify the torrent even though the .torrent file with the hashes it is comparing against was generated from the same file, though this won't be massive assuming you're source is at a low bitrate (10Mbps), this is planned to be fixed none the less.

What you're doing sounds quite interesting,
All the best.