iOS/Apple, Surface Pro - not able to receive the stream (the root cause provided)

[greko6]

Your environment.

• Browser: Safari 13.1.2 with Chrome on Windows
• Other Information - see below

What did you do?

1. Opened Chrome on Windows and started the stream.
2. Opened Safari or Chrome on macOS and couldn't receive the stream.

What did you expect?

See the stream.

What happened?

I couldn't see it.

After ionorg/ion-sdk-js#28 fix, I was able to experiment a bit more with Safari.

It turns out, that in the SDP Offer Safari has:

(...)
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 125 104
(...)
a=rtpmap:96 H264/90000
(...)

but in the SDP answer we have:
m=video 9 UDP/TLS/RTP/SAVPF 96
(...)
a=rtpmap:96 VP8/90000

For Safari, to get VP8/90000, the SAVPF should be set to 100.

96 works flawlessly with Chrome (on other OS than Mac/iOS), as with Chrome VP8/90000, indeed, it is 96.

To fix that, for maximum compatibility in the future, we probably need to look-up VP8 in the SDP Offer and return the same value to the browser.

Potential solution/fix/help here: pion/webrtc#716 (comment)

[greko6]

Also, this is the reason for the change: https://groups.google.com/forum/#!topic/discuss-webrtc/Z5YOWMC8TwU

It looks like Safari is using WebRTC before M57. Hence the reason why 100, but not 96.

Key quote from that thread: "External apps need to update their code to not rely on hardcoded values. This is all in accordance with the spec, payload types in the range 96-127 are dynamic and are not tied to any specific codec."

Most likely types will change again in the future.

[sibis]

@greko6 this issue exists iOS platform mobile and iPad in general. Since even chrome in iphone and ipad uses WebKit rendering engine not V8 just the chrome UI. So any app built on ion-sdk-js is not working on iphone and ipad irrespective of the browser

[sibis]

@tarrencev are we updating the docker image anytime now to include this fix?

[greko6]

I did compile it and created a new docker for my own SFU for testing purposes - after the patch was applied.

Unfortunately, SFU is still not matching it. Here is what I can see:

Offer:

Answer:

The answer should be:

m=video (...)
(...)
a=rtmpmap: 100 VP8/90000

With 96 Safari is still interpreting the video track as h264.

[greko6]

Surface Pro tablet is also affected by the same bug. As it's not fixed, can you reopen it, please?

[greko6]

@tarrencev could you re-open this issue, please? That way people that are using sfu will have much better visibility as it's not working.

[Sean-Der]

Hey @greko6 if you go to https://github.com/orgs/pion you should have an invite to join the org!

If you aren't able to join I will re-open, but hard for me to find bandwidth to help with Ion day-to-day :(

[greko6]

@Sean-Der : This issue, once solved, will allow pion to drastically accelerate as currently, people that are testing it with MacOSes/iPhones/iPads are not getting the streams at all. More recently, I've learned that also Microsoft Surface Pro is affected because of the same simple mismatch reasons.

[greko6]

Update based on the Slack thread.

TL;DR most likely we'll need to wait for v3

From Jeremija:
"The main issue here is that in the current version of pion/webrtc the PayloadTypes are defined in the webrtc.MediaEngine and are usually either populated from SDP or defined before any tracks are added. Populating from SDP does not scale well if you need to dynamically add tracks after the first negotiation.
The easiest solution for this is to generate the offers on the server-side and then the browsers will use server-defined PayloadTypes in their answers.
I think there are plans for v3 to fix this behavior."

[tarrencev]

this is working now but only on ion-sfu not with ion. you can verify it by running ion-sfu with the pub-sub-in-browser example. you will need a reverse proxy for ssl though since safari doesn't like ws

[greko6]

@tarrencev : Nice! Did you receive a stream on Safari? Which diff fixed it?