Simulcast

[Sean-Der]

Add WebRTC Simulcast Support

[mpisat]

How can we test this? I downloaded latest build from artifacts but I don't see any new option on OBS? A little help? :)

[Sean-Der]

Thanks for trying it out @mpisat!

I described how to use it here

I am adding support for Advanced Output Simulcast today. I will make sure to post updates on this PR as I work on it.

[llspalex]

Tested with Broadcast Box - works great! This is very cool indeed.

[ygit]

Nice! 🥇

[chhofi]

Hey @Sean-Der thanks for your great work! I am running the Arm MacOs OBS version and can successfully stream with simulcast enabled to the website. At least that is what I think is happening as all the metrics look good in OBS. On the website I also get the simulcast controls but unfortunately the stream does not load whether I use Safari or Chromium as a browser. What is the best way to debug this further and provide feedback?

[SetoKaiba]

Hi @Sean-Der , I got the same problem as @chhofi .
I tried it with siobud. And this is my settings.

It's not loading.

It does show the quality levels.

When I click stop, it shows this. Is it intended?

[Sean-Der]

Sorry about that @SetoKaiba @chhofi

What encoder are using? Can you try these settings? Does it help if you lower the bitrate? Does it work without Simulcast enabled?

[SetoKaiba]

Hi, @Sean-Der .
I use NVIDIA NVENC H.264 before.
I tried with your settings. It's sometimes able to load with green or blur pixels and extra low framerate, maybe 1-5.
But I can't determine whether this is caused by network.

And then I tried it with my local ZLMediaKit server. It's working with OBS WHIP before. It supports simulcast.
I tried with the translate-libdatachannel-to-cpp branch. It's working without simulcast.
I tried with the simulcast branch. It's not working even without simulcast.

And here's the warning I saw from ZLMediaKit logs for simulcast branch.
I didn't enable simulcast.

2023-08-06 16:34:53.251 W [MediaServer] [667-event poller 0] WebRtcTransport.cpp:779 onRtcp | 未识别的sr rtcp包:version:2
padding:0
report_count:0
pt:200(RTCP_SR)
size:28
--------
ssrc:1043019512
ntpmsw:3900299693
ntplsw:1079742464
ntp time:2023-08-06 16:34:53.251
rtpts:3000
packet_count:0
octet_count:0

"未识别的sr rtcp包" means "unrecognized sr rtcp packet".

And also, I found that the stream is registered as _l _m _h in ZLMediaKit even I didn't enable simulcast. Is the checkbox working?
It seems the change of simulcast checkbox need restart of OBS. May be it's a bug.

[SetoKaiba]

Here's the screenshot for using your settings with your server.
And also, I checked the chrome://webrtc-internals with choosing the quality level.
It someimes can be switched to other level. I have never switched to h level.

[Sean-Der]

@SetoKaiba can you try running https://github.com/glimesh/broadcast-box locally?

Just to debug

• is issue OBS encoding
• Is issue network
• Is issue WebRTC implementation

[SetoKaiba]

@Sean-Der I tried with broadcast-box go run . locally. It's working without problem.

Another small question, I run it with docker in windows. The video is not loaded. But the streaming doesn't report any error.

For ZLMediaKit, I'll try to debug the problem with ZLMediaKit's maintainers. But the ZLMediaKit is working with simulcast. Its demo page run successfully. So it seems to be a compatiblity problem.

Thank you.

[Sean-Der]

@SetoKaiba thanks for all the testing!

My guess is that we aren't implementing something properly? If you can figure out what ZLMediaKit is looking for I can implement that right away!

[SetoKaiba]

@Sean-Der Finally find out the problem with ZLMediaKit. The ZLMediaKit doesn't support mid. But this Simulcast implementation will always send the mid. He claims that mid is not part of the spec of Simulcast. The streaming should follow the answer.

[Sean-Der]

@Sean-Der if you delete this line https://github.com/Sean-Der/obs-studio/pull/2/files#diff-ae779f158b6d305fd42fd223123fa37e08cbd282a8fd479fc17447eff7c65a16R205 (and don't send MID) Simulcast then works for you?

[SetoKaiba]

@Sean-Der if you delete this line https://github.com/Sean-Der/obs-studio/pull/2/files#diff-ae779f158b6d305fd42fd223123fa37e08cbd282a8fd479fc17447eff7c65a16R205 (and don't send MID) Simulcast then works for you?

Yes. Or not using the directProxy of the ZLMediaKit, it will reassemble the rtp packet without extension header. Then it will work. He did a small fix now to filter the ext by answer instead of offer. It works also.
Anyway, he said that the packetization should follow the sdp exchange answer.