Application crashes when connecting with Safari

[birme]

0:01:51.712577629     1 0x7fec68015640 ERROR              webrtcice gstwebrtcice.c:316:_resolve_host: Error resolving “a64f400a-abbf-481e-8ee6-3676dfb9c509.local”: Name or service not known
0:01:51.713538447     1 0x7fec68015640 ERROR              webrtcice gstwebrtcice.c:316:_resolve_host: Error resolving “a64f400a-abbf-481e-8ee6-3676dfb9c509.local”: Name or service not known
**
ERROR:../ext/webrtc/gstwebrtcbin.c:2304:sdp_media_from_transceiver: assertion failed: (trans->mline == -1 || trans->mline == media_idx)

[Wkkkkk]

Hi @birme , I found that Safari prefers h264 of profile constrained-baseline.
To verify that, we could generate a local test stream.

gst-launch-1.0 -v \
    videotestsrc ! clockoverlay ! video/x-raw, height=360, width=640 ! videoconvert ! x264enc tune=zerolatency ! video/x-h264, profile=constrained-baseline ! mux. \
    audiotestsrc ! audio/x-raw, format=S16LE, channels=2, rate=44100 ! audioconvert ! voaacenc ! aacparse ! mux. \
    mpegtsmux name=mux ! queue ! srtsink uri="srt://127.0.0.1:1234?mode=listener" wait-for-connection=false

And then

GST_DEBUG=1 cargo run --release -- -i 127.0.0.1:1234 -o 127.0.0.1:8888 -p 8000 -s caller | bunyan

[Wkkkkk]

Here is a screenshot of it running.

[birme]

Ah, that's good to know.
Still the main issue now is the #32 which I think happens every time the app tries to remove a client connection.

[Wkkkkk]

That's true. We still need to fix that.

[Wkkkkk]

Some updates on this issue:

• This problem occurs when we connect to an SRT stream with main or high H264 video profile. Media line exists in the answer but is not included in BUNDLE and ice candidates exchange will time out (similar to issue Support H.265 / HEVC #26).
• One possible workaround for H264 video is to transcode the stream into a supported profile before connection (as in issue Need to support different video profiles #36). However, this is not recommended as it is computing expensive and it is not possible for H265 video on Mac.
• A better fix is to look into the answer generated by browsers and understand the cause for missing media lines.

[birme]

Also, one rationale not to transcode the video stream in this component is that we want to maintain the video as it is as one use case is confidence monitoring or preview. It is ok for now to limit supported browsers depending on the actual SRT video stream. However the component should not crash or end up in an unresolvable state.

[Wkkkkk]

This has been addressed in the recent commit so this becomes less of a issue now.

However, it is still a problem that Safari rejects AVC/H264 streams of main or high profile. This relates to how Safari parses the codecs parameters (e.g., profile-level-id) and more test will be done.

[birme]

Alright, good to know.
We should perhaps document all the known limitations somewhere?

[Wkkkkk]

While experimenting with Safari and AVC/H264 streams, it is noticed that Safari only supports WebRTC H264 of constrained-baseline and constrained-high profiles. This can be seen from the SDP offer it generated (either with webrtc player in client mode or with other tools like MungeSDP).

a=rtpmap:96 H264/90000
...
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f
a=rtpmap:98 H264/90000
...
a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

Here profile-level-id can be parsed into constrained high level 3.1 and constrained baseline level 3.1 respectively, as it is defined in the WebRTC repository. Unfortunately, neither ffmpeg nor gstreamer could generate video streams of constrained high profile. So we instead tested with one stream of constrained baseline and one of high (munging the profile-level-id from '64001e' to '640c1f' in SDP offer).

As we have seen before, constrained baseline video stream can be played out successfully on Safari. On the other hand, high video stream will be rejected. But if we munge it into '640c1f', the connection can be established but no stream is actually played out.

With the findings so far, a document about these limitations will be drafted and included in this repo soon.