-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add WebRTC (WHIP) output support #7926
Conversation
FAQHow can I use this today?Multiple companies provide WebRTC Broadcast/Playback today. Cloudflare and Millicast are two popular options. Gilmesh has created and is hosting a public WebRTC server at https://b.siobud.com/. You can run this locally for developing and testing as well https://github.com/glimesh/broadcast-box Why libdatachannel as the WebRTC library?WebRTC has many libraries that we could have used. The first one we evaluated was Google's implementation known as libwebrtc. Fetching the source of libwebrtc: 11 minutes 20 seconds This caused us to evaluate other options. We arrived at https://github.com/paullouisageneau/libdatachannel and it had everything we needed! It is a C++ WebRTC implementation and uses CMake. It adds very litle impact to the project. Additional Build time: 9 seconds What is WHIP?WHIP is the process that WebRTC uses to establish a session between two WebRTC Agents (Client/Server in our case). The actual definition of it can be found at https://www.ietf.org/archive/id/draft-ietf-wish-whip-05.txt |
06925eb
to
18dcd1d
Compare
e8a10c8
to
218663b
Compare
This comment was marked as spam.
This comment was marked as spam.
6994c0d
to
75c1acf
Compare
This adds a new OBS_SERVICE_CONNECT_INFO_BEARER_TOKEN which is needed for WHIP output. Signed-off-by: pkv <pkv@obsproject.com>
This adds a WHIP output & associated service. - Code inspiration from DDRBoxman - Implemented by Sean DuBois & tt2468 - Various fixes and contributions by pkv. Co-authored-by: tt2468 <tt2468@irltoolkit.com> Co-authored-by: DDRBoxman <colin@recursivepenguin.com> Co-authored-by: pkv <pkv@obsproject.com> Signed-off-by: pkv <pkv@obsproject.com>
This provides the UI glue to enable the WHIP service introduced in the obs-webrtc plugin. Co-authored-by: John Bradley <jocbrad@twitch.tv> Signed-off-by: pkv <pkv@obsproject.com>
a6c84b2
to
9925427
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some reason, |
That's an oversight |
It works well, but periodically the image is showered with artifacts (tested with NVENC, QSV and x264). Great job, I'm waiting for support for B-Frames and other codecs |
After the commit dd39218, obs gets crashed when opening the settings dialog. Steps to reproduce is as below.
Log shows a line below and obs crashed immediately.
Backtrace is as below.
|
Sorry about that @norihiro i will have that fixed today! |
In general, we prefer that the deps are the same version across all platforms whenever possible so that we aren't guessing at behavior differences. This can be done separately as a one-line change. paullouisageneau/libdatachannel@v0.19.0-alpha.1...v0.19.0-alpha.3 |
Yeah, i was just suggesting that i'd make a new change to the commit which is checked out, before the first beta, in order to remove the current libdatachannel patch so maybe it's not necessary to multiply the PRs ? Your call |
Personally, I'd prefer the deps were aligned sooner rather than later. Less time for people to run into obscure issues. Want CI builds to be in the best shape we can manage, and a one-line PR is simple enough to review. |
|
@fairbairn What encoder (and settings) are you using? I saw this latency when I did a |
x264, main, 5000K, CBR, 1080p, 30fps, ultrafast, zerolatency did you use main or baseline? also, were you streaming audio as well, or just video? |
We have been unable to replicate this latency, even locally while transmitting through broadcast-box to remove all network influences. We're unable to get below 210-260ms in our various tests, and we always tune for low latency on the video encoder by default (x264 as well as NVIDIA). Can we get the precise setup you used (audio and video) from your test where you achieved 120ms so that we may reproduce in our own lab? In our tests, we're accompanying the video with audio, which may have an influence, but we'd very much like to reproduce your baseline before we tune from there. Thank you. |
Hi @SetoKaiba I have a build of Simulcast available here I am actively working on it so will have bugs, thank you! |
Sorry I missed this! It might be better if you reach out on OBS discord. I get a lot of github notifications, easy to miss things :/
|
RFC for WHIP: https://datatracker.ietf.org/doc/draft-ietf-wish-whip/ RFC for WHEP: https://datatracker.ietf.org/doc/draft-murillo-whep/ Please note that SRS 5.0 already had WHIP support. I didn't write a document about WHIP, because WHIP is not a RFC right now, but there are clues in [srs-unity](https://github.com/ossrs/srs-unity#usage-publisher). SRS WHIP url for publisher: `http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream` This PR is for WHEP, the url for player is `http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream` PS: There is a great PR for OBS to have WHIP support, see obsproject/obs-studio#7926 and #3581 PS: WHIP for FFmpeg ossrs/ffmpeg-webrtc#1 See #3170 --------- Co-authored-by: Haibo Chen <495810242@qq.com> Co-authored-by: john <hondaxiao@tencent.com> Co-authored-by: ChenGH <chengh_math@126.com>
RFC for WHIP: https://datatracker.ietf.org/doc/draft-ietf-wish-whip/ RFC for WHEP: https://datatracker.ietf.org/doc/draft-murillo-whep/ Please note that SRS 5.0 already had WHIP support. I didn't write a document about WHIP, because WHIP is not a RFC right now, but there are clues in [srs-unity](https://github.com/ossrs/srs-unity#usage-publisher). SRS WHIP url for publisher: `http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream` This PR is for WHEP, the url for player is `http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream` PS: There is a great PR for OBS to have WHIP support, see obsproject/obs-studio#7926 and #3581 PS: WHIP for FFmpeg ossrs/ffmpeg-webrtc#1 See #3170 --------- Co-authored-by: Haibo Chen <495810242@qq.com> Co-authored-by: john <hondaxiao@tencent.com> Co-authored-by: ChenGH <chengh_math@126.com>
RFC for WHIP: https://datatracker.ietf.org/doc/draft-ietf-wish-whip/ RFC for WHEP: https://datatracker.ietf.org/doc/draft-murillo-whep/ Please note that SRS 5.0 already had WHIP support. I didn't write a document about WHIP, because WHIP is not a RFC right now, but there are clues in [srs-unity](https://github.com/ossrs/srs-unity#usage-publisher). SRS WHIP url for publisher: `http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream` This PR is for WHEP, the url for player is `http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream` PS: There is a great PR for OBS to have WHIP support, see obsproject/obs-studio#7926 and ossrs#3581 PS: WHIP for FFmpeg ossrs/ffmpeg-webrtc#1 See ossrs#3170 --------- Co-authored-by: Haibo Chen <495810242@qq.com> Co-authored-by: john <hondaxiao@tencent.com> Co-authored-by: ChenGH <chengh_math@126.com>
Co-authored-by: Sean DuBois sean@siobud.com
Co-authored-by: DDRBoxman colin@recursivepenguin.com
Co-authored-by: tt2468 tt2468@gmail.com
First and foremost, thanks to the original author of the draft PR for webrtc Colin Edwards (DDRBoxman).
Description
This PR adds WebRTC support to OBS. WebRTC is an alternative to RTMP that is being adopted by more services. Adding WebRTC support to OBS would benefit OBS and its users greatly. These are some of the highlights. Following this I also have a FAQ for questions we have already received regarding this PR.
This initial version is intentionally simplified to only support output. Later PRs will add source support.
Specifically, this adds a plugin that extends support for:
Motivation and Context
Subsecond Latency
With WebRTC we are able to achieve sub second latency. With our initial measurements we see ~120 Milliseconds from Broadcaster to Playback. We believe we can continue to bring down this number as well.
Interactive latency lets broadcasters create new experiences they couldn’t before. OBS users can now create interactive experiences with their viewers. This would allow talk shows and other productions that require conversational latency to use OBS.
Remove FTL from OBS
FTL is a protocol that was created for Mixer. It provided sub-second latency and was used by a few Open Source projects and streaming services. The protocol has been abandoned and it would be nice to remove it from OBS. Adding WebRTC would make that possible since it provides the latency required in addition to the many benefits inherent in the WebRTC stack like encryption, network topology strategies, robust congestion control, etc.
Latest in Video Compression
WebRTC will let OBS users experiment with newer codecs. WebRTC is flexible in what codecs it supports so users could use H265 and AV1.
AV1 users are reporting a 50% reduction in bandwidth required.
Broadcast all angles
WebRTC supports multiple video streams in the same session. OBS could experiment with broadcasting multiple scenes at once! Viewers could then switch between scenes or different views in the same game.
Simulcast
WebRTC allows broadcasters to upload multiple streams of different quality. OBS users could upload ‘high’, ‘med’, and ‘low’ streams themselves. This would reduce the server costs greatly, allowing more
Peer-to-Peer (if you need it)
With WebRTC OBS users could share their video without needing a video server at all! WebRTC can establish a P2P connection so users could send video from OBS directly to their users. No more worrying about setting up special servers to ingest and transcode.
In the future we could add a WebRTC source. Then broadcasters could add WebRTC sources to their scenes. This means low latency collaboration with no backend required!
How Has This Been Tested?
This has currently been tested on Linux and Mac with locally run broadcast-box as well as remotely hosted WHIP endpoints.
Configurations tested:
Original draft PR: #7192
Closes #7192
Types of changes
Checklist: