Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebTorrent support #4123

Merged
merged 12 commits into from Jul 1, 2020
Merged

WebTorrent support #4123

merged 12 commits into from Jul 1, 2020

Conversation

paullouisageneau
Copy link
Contributor

@paullouisageneau paullouisageneau commented Nov 25, 2019

This PR suggests an implementation of the WebTorrent protocol inside libtorrent. With this add-on, libtorrent can be used to run hybrid peers seamlessly.

WebTorrent peers use the same application protocol as classic peers but with WebRTC Data Channels as underlying transport instead of uTP or TCP, allowing to run peers in web browsers. Due to the necessary connection signaling, WebTorrent peers require a specific WebSocket tracker to connect.

Since documentation seems to be lacking, I implemented the WebTorrent signaling protocol the best I could guess by looking at existing code. You can find useful information and the reference Javascript implementation here: https://github.com/webtorrent/webtorrent

The implementation requires two external dependencies added as submodules:

  • libdatachannel, my lightweight WebRTC Data Channels implementation, which relies on my ad-hoc UDP Interactive Connectivity Establishment (ICE) library libjuice
  • boost.JSON, a header-only JSON library for the tracker protocol

This PR does not add new external dependencies to libtorrent (it only requires OpenSSL or GnuTLS depending on the compilation option).

To test, you need a torrent file with at least one WebSocket tracker, for instance wss://tracker.openwebtorrent.com.
You can find such a torrent file here: https://ageneau.org/upload/sintel.torrent
You can also run a web browser peer on this page: https://webtorrent.io/

include/libtorrent/torrent.hpp Outdated Show resolved Hide resolved
src/torrent.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/websocket_tracker_connection.cpp Outdated Show resolved Hide resolved
src/torrent.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
src/rtc_stream.cpp Outdated Show resolved Hide resolved
@alexeyvolkoff
Copy link

@alexeyvolkoff alexeyvolkoff commented Aug 11, 2020

@paullouisageneau
This checking is just for logging.
I think checking should be like this:
if (m_connecting && m_peer_info && t) t->inc_num_connecting(m_peer_info);
But is it OK to have dead connection? Question to @arvidn

@arvidn
Copy link
Owner

@arvidn arvidn commented Aug 11, 2020

I does look suspicious. I don't have time to investigate this right now. the comment in the header file says that m_peer_info can be nullptr for incoming connections, where we don't know which torrent it belongs to yet. which suggests that if t is non-zero, then m_peer_info should be non-zero as well.

@alexeyvolkoff
Copy link

@alexeyvolkoff alexeyvolkoff commented Aug 11, 2020

with my fix it is just freezing.

@alexeyvolkoff
Copy link

@alexeyvolkoff alexeyvolkoff commented Aug 11, 2020

@arvidn Now it works better but freezes on exit.
libtorrent/src/session.cpp:451, on m_thread->join();

BTW is the master branch stable?

@arvidn
Copy link
Owner

@arvidn arvidn commented Aug 11, 2020

It depends on what you mean by stable. Clearly it's not stable enough :)
but it's not like we land patches that fail CI. I imagine the issue you're experiencing is some interaction with the new webtorrent feature

@alexeyvolkoff
Copy link

@alexeyvolkoff alexeyvolkoff commented Aug 13, 2020

Very interesting. Now in another place:
libtorrent/src/torrent.cpp:7232:29: runtime error: member access within null pointer of type 'struct torrent_peer'
Again, it takes 6-7 hours to reproduce. Client is mostly idle.

@xavier2k6
Copy link

@xavier2k6 xavier2k6 commented Aug 19, 2020

@arvidn I know this has been merged in to master but was just wondering if this is targeted to be released for RC_1_x , RC_2_x or some other branch down the road like RC_3_x?

@arvidn
Copy link
Owner

@arvidn arvidn commented Aug 19, 2020

some other branch after 2.0

@xavier2k6
Copy link

@xavier2k6 xavier2k6 commented Aug 19, 2020

some other branch after 2.0

Thanks.

@petterreinholdtsen
Copy link

@petterreinholdtsen petterreinholdtsen commented Sep 4, 2020

Do you have any idea when the webtorrent support will be included in a release? Any chance it will happen in time for the Debian release freeze around new years eve?

@arvidn
Copy link
Owner

@arvidn arvidn commented Sep 5, 2020

it will not have been released by end of year. It's not quite stable yet. For example, this issue has come up: #5100

I welcome anyone to test libtorrent master to iron out issues like that, to get it to a stable state sooner!

@frankenstein91
Copy link

@frankenstein91 frankenstein91 commented Dec 24, 2020

is there any info on which clients are already working with it (webtorrent)?

@tuxayo
Copy link

@tuxayo tuxayo commented Dec 25, 2020

It's not released yet, more info in the previous messages.

@Clodo76
Copy link

@Clodo76 Clodo76 commented Jan 9, 2021

I hope in progress of this integration, this world need an implementation to share web-based p2p content.
i just donate 1000€ to the bitcoin address of libtorrent project, i hope can help.

@paullouisageneau
Copy link
Contributor Author

@paullouisageneau paullouisageneau commented Jan 28, 2021

@Clodo76 That's very nice of you, however for now I'm the only one maintaining the integration. Any contribution to help to test and fix issues (for instance #5831) is welcome!

@gubatron
Copy link
Contributor

@gubatron gubatron commented Apr 20, 2022

is webtorrent support built as a default feature, or does it need to be specified with some build flags?

Can't find anything in Build Features or any examples on how to enable webtorrent.

@paullouisageneau
Copy link
Contributor Author

@paullouisageneau paullouisageneau commented Apr 20, 2022

@gubatron Webtorrent support is present only on master branch for now, not on RC_2_0 branch, and it requires the build feature webtorrent to be enabled

target_optional_compile_definitions(torrent-rasterbar PUBLIC FEATURE NAME webtorrent DEFAULT OFF

@petterreinholdtsen
Copy link

@petterreinholdtsen petterreinholdtsen commented Apr 20, 2022

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet