Bring torrents to the browser (chrome extension)

[ghost]

Hi everyone

I'm new on this git, but I just wanted to share my idea and see your reactions.

So, what if we created/used a chrome/Firefox extension that downloads and seeds torrent files over the bittorrent and webtorrent network. This way we can close the gap between these networks. And make a 100% browser side torrent client.

From a user's perspective:

• goes to website
• wants to download torrent/play video
• a) torrent file is available on webtorrent network
  -> starts direct download
  -> seeds to webtorrent network
• b) torrent file is not available on webtorrent
  -> is asked to install extension
  -> starts to download
  -> seeds to webtorrent network (&bittorrent)

[ghost]

Does anyone think this is a good idea, is realistic and/or has any value to this project?

[Stiveknx]

I think it is an great ideia!
It would help a lot!

Altought I think we would need some ideias to share the extensions to another users, I mean, why they would change from their current clients to another one?

Also, two another good ideias I've to solve the gap:
A desktop client (can be built with nw.js).
A tool to download files on a seedbox / dedicated server (from udp peers), and share it with webtorrent clients.

This second ideia, I started a project a few months ago, and it's working fine, for my tests.

[madd512]

https://github.com/feross/chrome-net relevant?

[iadj]

+1 for this idea!

[ericwooley]

We made a working chrome extension for webtorrent for a hackathon, it worked pretty well https://github.com/RosettaLabs/Torrentula (webRTC only). Something changed with the chrome api and local storage and it doesn't run anymore, should be a fairly easy fix though if you guys want to fork it or canabalize it. It has a fully functional add/delete/view torrent manager built in react with a flux architecture.

[madd512]

That's interesting. I think the great thing about using a chrome app for this, though, would be access to the primary bittorrent network.

[ghost]

Thank you for the comments, I'm currently trying to figure out how exactly everything is working.
I think we first should start creating a hybrid client (if this doesn't exist yet).
After that we can try to implement this in a chrome extension.

My current questions:

• is there already a hybrid client?
• is ther a difference between webtorrent and bittorrent torrent files/ magnet links (if this is the case how do we link them)?
• how do clients find each other?
• where/how are the torrent files getting stored?

Also I found this project:
https://github.com/mafintosh/torrent-stream
This project is very simular to this project but uses bittorrent, if there is no hybrid client yet we can use this project to create one.

[ericwooley]

is ther a difference between webtorrent and bittorrent torrent files/ magnet links (if this is the case how do we link them)?

No, there is no difference, the issue is that webtorrent (in browser) is limited to webRTC, and other clients haven't implemented that connection method

how do clients find each other?

There are two methods of finding peers, a tracker, or a DHT (Distributed Hash Table). You can lookup how either work

where/how are the torrent files getting stored?

I am still not exactly sure about how it is stored in browser. It might be in memory if you download it. This is used, https://github.com/mafintosh/memory-chunk-store but I don't think it does local storage or anything.

[gillesdemey]

Check out https://github.com/feross/webtorrent-hybrid for a bridge between BitTorrent/BitTorrent over WebRTC.

[madd512]

webtorrent-hybrid is just webtorrent with a native wtrc module. Since chrome apps can already do wrtc, I think just net needs to be polyfilled, which is why I posted https://github.com/feross/chrome-net. It says it's used by webtorrent, but it isn't. This makes me think that @feross tried to get webtorrent to work in a chrome app recently.

I think building webtorrent with browserify, substituting chrome-net for net, might work. Somebody should try it.

[ericwooley]

@devtristan I think you are mistaken about webtorrent-hybrid, it connects to both webrtc and TCP/UDP peers. Webtorrent (non-hybrid) is only for webrtc in the browser, and IIRC only tcp/udp in node.

So you need to use webtorrent hybrid for both. Webtorrent will only do one, or the other, depending on the enviornment.

[madd512]

Right, sorry, that's what I meant. See the source code:

https://github.com/feross/webtorrent-hybrid/blob/master/index.js
https://github.com/feross/webtorrent-hybrid/blob/master/lib/global.js

I'm not sure how webtorrent decides to connect to the main bittorrent network or not, but I don't see a reason why it wouldn't be able to, and bittorrent-hybrid wouldn't work due to the native module.

[madd512]

Looks like net availability is determined here at least, possibly elsewhere... https://github.com/feross/bittorrent-swarm/blob/master/index.js#L320

[DiegoRBaquero]

Great idea! +1

[jimmywarting]

+1
Its definitely doable! Just tried torrent stream chrome extension

It uses bittorrent in a chrome extension. It can stream torrents and its also seekable witch is grate! (think the CPU went up when i tried)

[madd512]

I managed to get this working a while ago. It's an MVP but it runs.

https://gist.github.com/devTristan/b768d2ee571350d5b796

The hard part was aliasing all of the node-only modules and overriding browser excludes. It was pretty easy to find chrome-compatible modules to polyfill things like net and dgram, because @feross already made them! I was also able to grab native-dns to polyfill dns. That just uses the net module provided by chrome-net and gets DNS resolving using the chrome APIs.

What works:

• Downloading and seeding on the bittorrent network
• Downloading and seeding on the webtorrent network (it's a hybrid client!)
• The DHT. My favorite data structure. Thanks, chrome-dgram!
• Downloading totally legal torrents and embedding them in video tags. Thanks, blender open movie project!

What doesn't work:

• Anything after about 3 minutes. It crashes either the app or the entire browser. Pretty neat, huh?

I don't know why it crashes. I suspect it's something to do with trying to allocate too much in the way of resources - memory, disk space, network connections, etc. When I do profiling, it looks like the JS-Native bridge is the bottleneck with all of the network api calls, but I could be wrong.

You should be able to clone my gist and try it out yourself. Happy hacking.

[loon3]

I'm having the same issue with everything working fine for a couple minutes then getting bogged down. I can slowly watch the CPU utilization tick up as well as the Packets in/out. My guess is it's slowly duplicating processes but I can't figure out why.

This is my background script... https://github.com/loon3/Tokenly-Pockets/blob/master/Chrome%20Extension/js/webtorrent/bvam-seed.js

[eliooses]

Hi @devtristan !,

I try to run your extension, but i can makes work.

I put the bundle.js (not include in your package at same of app.css) ignore the app.css and put a Magnet for test and the code retrieve the follows erros:

Failed to load resource: net::ERR_FILE_NOT_FOUND
3extensions::platformApp:17 window.localStorage is not available in packaged apps. Use chrome.storage.local instead.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.toolbar is not available in packaged apps.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.statusbar is not available in packaged apps.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.scrollbars is not available in packaged apps.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.personalbar is not available in packaged apps.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.menubar is not available in packaged apps.(anonymous function) @ extensions::platformApp:17
2extensions::platformApp:17 window.locationbar is not available in packaged apps.

[madd512]

Hey @eliooses, did you use the npm run watch or npm run watch-js script to kick off your build?

I think all of those warnings are fine, but the net::ERR_FILE_NOT_FOUND error might be related to the aliasing of the fs module as base:/fakefs.js. Make sure fakefs.js is in the same directory as your package.json.

If that doesn't help, try throwing it all in a github repo and I'll clone it and try it out.

[elad]

This idea - torrent client as a Chrome extension - was actually one of the first things I did when starting to play with webtorrent. It was very easy to get working. If anyone's interested in the code, I can upload everything I have as-is to a repository.

[professorhaseeb]

@aqwsez blank repo!!!

[fontanierh]

Is anyone still working on something here? I would like to contribute. Would be awesome to be able to download regular torrents from the browser and seed them on webtorrent

[knoopx]

FYI:

Important: Chrome will be removing support for Chrome Apps on Windows, Mac, and Linux. Chrome OS will continue to support Chrome Apps. Additionally, Chrome and the Web Store will continue to support extensions on all platforms. Read the announcement and learn more about migrating your app.

https://blog.chromium.org/2016/08/from-chrome-apps-to-web.html

[HughIsaacs2]

To bring this back alive, something I'd like to add.

Building support for the Chrome UDP Sockets API in WebTorrent should also allow us to build WebTorrent support in Android apps via Google's sockets.udp Cordova plugin https://github.com/MobileChromeApps/cordova-plugin-chrome-apps-sockets-udp and even though Google is dropping support for Chrome Apps on Windows, Mac and Linux, I'd argue that Chrome OS support is still important considering that the userbase on that end keeps growing.

Also, a question, how viable would it be to build this extension using the Chrome Native Messaging API?
https://developer.chrome.com/extensions/nativeMessaging

I'm thinking it might be a good idea to just have a Chrome extension silently open the WebTorrent Desktop app and start downloading the torrent (with some sort of permission prompt first for security reasons of course) and then have it seed to the web page via WebRTC.

I'm no expert, but it shouldn't be too hard to build this 2nd option, shoot I'll gladly build this myself. I just need to know if this would work.

[devmattrick]

I'm a bit late to the party here, but it would be nice to have the extension also give sites using the WebTorrent API the ability to use the raw sockets provided by browser extensions (it seems to currently only be supported by Chrome, but there is a proposal to add it to the WebExtensions spec). This would allow WebTorrent clients to be able to access the entirety of the BitTorrent network without clients supporting WebRTC needing to be available.

I'm not entirely sure how the client <-> extension communication would work, but I imagine there are a number of ways of going about it. One possible way is to somewhat replicate how the Vue DevTools extension does it; it adds a property to the window which is essentially an EventEmitter bridge between the extension and client script.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[feross]

FWIW, I think a WebTorrent Chrome App is a great idea for Chrome OS users. I think that it won't help Mac, Windows, and Linux users much since Chrome Apps are deprecated and support will eventually be removed.

Unfortunately, the chrome.sockets API is only available in Chrome Apps not Chrome Extensions, so we can't make a Chrome Extension that allows full torrent network connectivity including over TCP and UDP.

For that reason, I personally don't consider making a Chrome App to be a top priority but if someone wants to take a stab at making one, go for it.