Support Apple Safari

[mccob]

Apple Safari/Webkit support is here since Safari 11.

Apple use Google/WebRTC libwebrtc.

See :
https://webkit.org/blog/7726/announcing-webrtc-and-media-capture/
https://webkit.org/blog/7627/safari-technology-preview-32/

Available at least on Mac OS.

Safari tech preview is available for actual Mac OS Sierra.

Licode isn't working.

Seems a specific stack is needed.

First error solved, has to use window.RTCPeerConnection

WebkitRTCPeerConnection = window.RTCPeerConnection,

Other errors that must be solved :

• [Error] Unhandled Promise Rejection: TypeError: Type error createObjectURL
• [Error] TypeError: Error creating RTCPeerConnection initializeRTCPeerConnection
• [Error] TypeError: undefined is not an object (evaluating 'stream.pc.processSignalingMessage')
  (anonymous function)

if you want to help :)

[kekkokk]

I'll probably look into it next week! have a nice day!

[mkhahani]

Finally!!
It's a good time for migrating to adapter.js and get rid of all these browser stacks.

[kekkokk]

We might discuss that, is not that straight forward. we have always to differentiate by chrome and firefox for little things (ex. opus nack, the "a=" in front of ices, the filtering out of the end candidate in chrome etc.) so maybe we'll always have the various stacks.
but with a refactor of the sdp parser in C++ we might have good results with adapter.

[jcague]

I've been trying this week. I was able to create a videoconference with publishers sending video/audio from Chrome, Firefox and Safari, after solving all the errors mentioned by @mccob. Things to consider when we implement it:

• Safari only supports h.264.
• h.264 support in Licode is not perfect, we might need to add better support for fmtp lines in the SDP. I found some issue when sending H.264 from Firefox because of that, but I solved it with a quick and dirty hack.
• I was able to send video from Safari, and I received and rendered it in Chrome and Firefox.
• I didn't receive any data in Safari from publishers in Chrome and Firefox. The API is slightly different and I don't know if things like OfferToReceiveVideo or Transceivers are completely implemented in Safari's current implementation (WebRTC in Safari is still a WIP, but really close to be completed I think, and they're probably thinking of adding native support for WebRTC in iOS 11, I think I saw something about it in their last keynote).

Completely agree with @kekkokk, with might need to discuss adapterjs and/or other solutions. Anyway, thanks @mccob for playing with Safari. Can you try enabling h264 in the rtp_media_config?

[jcague]

FYI: master...jcague:poc/safari this is the result of a couple of hours trying to make it work... I'm not proud of it, it's plenty of hacks!! so please don't use it in production, staging or even debugging 😂

[kekkokk]

@jcague any plan to try edge? I opened the most detailed post i could @ DISCOURSE but I had no response :(
in business environments is way more used than safari.
Another question I never asked: is licode h264 ready?
I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264.
Do you think I could switch seamlessly into h264 without problems? or something (ex. externaloutput) will break? I never tried....

[menelike]

@jcague

h.264 support in Licode is not perfect

Are you talking about P2P oder the MCU mode?

Asking because we need h264 support on the MCU, because of

I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now* and observed performance problems, we'd love to try h264 since this might be a hardware support related issue.

*We use adapter.js and actually the chrome stack for iOS, works fine for a long time now (but we don't use the latest features like SimulCast, but Opus, Isac etc works fine!)

[thehappycoder]

Is there any ETA on when Licode starts supporting Safari? Even if it's not perfect but good-enough for some limited use cases.

[miniruwan]

Calls

I found the reason why safari doesn't receive media from other browsers.

I didn't receive any data in Safari from publishers in Chrome and Firefox.

MCU doesn't send answer sdp until it receives ICE candidates. But safari doesn't generate and send ICE candidates. https://bugs.webkit.org/show_bug.cgi?id=176157
One way to fix this is a shim in adapter.js (https://github.com/webrtc/adapter/issues/661)

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

Recording

For call recording to work, ExternalOutput probably needs to be changed to follow the h264 specs. Is there any ETA for this?

[mkhahani]

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

@miniruwan Could you get it to work with Licode? I built EC with the latest adapter.js (v6.3.2) but still could not receive audio/video in Safari.

[kekkokk]

try with this SafariStack

/* global */

import Logger from '../utils/Logger';
import BaseStack from './BaseStack';

const SafariStack = (specInput) => {
  Logger.info('Starting Safari stack');
  const spec = specInput;

  const newIces = [];

  spec.iceServers.forEach((server) => {
    const newServer = {};
    if (server.url) {
      newServer.urls = server.url;
    }
    if (server.username) {
      newServer.username = server.username;
    }
    if (server.credential) {
      newServer.credential = server.credential;
    }
    newIces.push(newServer);
  });

  spec.iceServers = newIces;

  const that = BaseStack(spec);

  return that;
};

export default SafariStack;

[mkhahani]

Thanks for the response but it didn't help :(

Hopefully in p2p mode, Chrome2Safari and Safari2Chrome work well on both MacOS and iOS. I believe it can be fixed to work also in MCU mode.

Even I think Chrome over iOS must be able to receive audio/video after Safari added support for WebRTC since Chrome uses WKWebView component for iOS and it supports RTCPeerConnection now.

[menelike]

added support for WebRTC

Just my two cents...

Well yes, they "support" webRTC but still, VP8 is not supported just to name one of the many drawbacks. Without going into the details on how flawed webRTC on iOS is, as long as you have to rely on mobile browsers between iOS and Android, results are unpredictable. Your best chance will be to stick to h264, but then licode doesn't support h264 within the MCU.

IMHO for a complete working support for all devices including desktop browsers, iOS and Android clients need to be implemented natively. In addition/or instead a transcoding MCU is advised.

[kekkokk]

actually Licode does support for h264. and works well with safari. I actually made lots of changes to be able to, mainly you can see my EDGE stack PR and add the stack I posted few comments above

[mkhahani]

@menelike
Thanks for your advice. I'm just trying to find a way to add support for iOS in my app and it's become a nightmare.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now

Actually I was going to write my own hybrid app using cordova+iosrtc plugin, as discussed here, but I figure it out that the provided iOSStack is out of sync with the latest licode changes. Do you have any update on that?

[mkhahani]

@kekkokk
Are you saying you could get the mcu to work with Safari+iOS? Then where can I find your PR?

[menelike]

@mkhahani
We have to use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc as well, and it's a pain. The project is almost dead, one major problem will be, that h264 playback will result in a green screen (h264 is the first choice on iOS because it should reduce the CPU load)

I think all work regarding webRTC has shifted from Cordova to react native (https://github.com/oney/react-native-webrtc), imho Cordova and webRTC will never work, especially not since getUserMedia is not available in WKWebView.

We also switched from licode to https://github.com/meetecho/janus-gateway, but this won't help you in your situation.

[kekkokk]

@mkhahani We actually have one dev only for iOS and we did a native app with the webrtc core.
the webrtc core supports vp8 on iphone.

[mkhahani]

@menelike

imho Cordova and webRTC will never work

But it was working in my tests before the latest changes in Erizo Client.

We also switched from licode to https://github.com/meetecho/janus-gateway

Do you really found Janus better than Licode?!

[mkhahani]

@kekkokk

We actually have one dev only for iOS and we did a native app with the webrtc core.

Wow that's a lot of work! What do you mean by the "webrtc core" exactly? Are you using the one provided by webrtc.org? And is that compatible with Licode?

[kekkokk]

yeah sure! that's a ton of works but, actually, is the one used by chrome. so the sdps are quite similar and supports vp8.

[mkhahani]

@kekkokk
@menelike
What do you think about Temasys WebRTC Plugin which is used by Jitsi to add support for WebRTC in IE and Safari?

[kekkokk]

we are using this only for IE and works quite well (produces sdp very similar to chrome).
For the safari side, we found very difficult, now that safari 11 supports natively webrtc, to interoperate with the plugin.
It was too difficult to differentiate safari versions and choose to use the plugin or not.
ATM we enabled only safari >= 11 in our platform so the call will init in h264 room

ps. doesn't support simulcast ofc

[zevarito]

@mkhahani There is an IOS ObjC client for Licode I've done a while ago, not sure how it works with latest version of Licode since I didn't had chance to work on it lately but I am sure if it doesn't work out-of-the-box it just need some signaling-protocol update almost. PR and maintainers are welcome, Link bellow in case you want to check it out.

https://github.com/zevarito/Licode-ErizoClientIOS

[mkhahani]

Thank you @zevarito, I am aware of your iOS app but I'm trying to avoid writing native since I've done a lot of work on other parts of my app like ui, signaling, etc.

[jaydgoss]

@mkhahani Did you find that Chrome to Safari, iOS to macOS combinations work in p2p mode?

[mkhahani]

@kekkokk
Interesting!! May I have your stack please?

[kekkokk]

Have you tried with your basestack? it should be work fine. I mean, my base stack (js) is very similar to the project one. If it does not work I suggest you to try to disable the single peerconnection feature and comment this line:

licode/erizo_controller/erizoClient/src/Stream.js

Line 38 in 4ddea91

if (evt.stream.id === that.getLabel()) {

If it does not work it's maybe something we changed in c++ side but I don't think

[mkhahani]

Does it mean I need to change the following code to use BaseStack?

licode/erizo_controller/erizoClient/src/ErizoConnectionManager.js

Lines 35 to 37 in 4ddea91

	} else if (this.browser === 'safari') {
	Logger.debug('Safari using Chrome Stable Stack');
	this.stack = ChromeStableStack(spec);

[kekkokk]

no chrome stack should be ok, it only shim the simulcast

[mkhahani]

I get the following errors when publishing/subscribing with Safari using VP8 while it works fine with H264:

[mccob]

update :

working without any update with last licode release (and Safari Tech Preview). Thank you @kekkokk and everyone. Hope Apple will provide this to iOS soon.

[lidedongsn]

publish from Safari ,sdp "a=sendrecv", Safari send all candidates, and licode acts "All candidates received", but there is not to continue, nothing going on! ICE state not ready. why?

[mkhahani]

@mccob
iOS 12 is out and it works using Safari in P2P mode! But I'm still getting the above error in MCU mode. What version of Licode are you using? Any customization?

[mccob]

No relation with VP8, iOS 12 was out before Apple announce VP8 support.

Licode can work with h264, in P2P mode only ? Because I already try in no P2P mode it's not working.

[mkhahani]

I see but I also get the error on Safari Tech Preview. How could you get it to work?

[mccob]

Do you enable vp8 support on preview options ?

[mkhahani]

Ah! No I never knew about that.

[kekkokk]

Great job Apple! once in a while

[mkhahani]

Yes it's working now. 👍
Thank you.

[lidedongsn]

safari only subscribe，no video play！
MediaElementSession::autoplayPermitted(112EA802) Returning FALSE because element is not visible in the viewport

[pawanvivekananda]

I am facing same issue in safari 12 on MAC

[mccob]

https://developer.apple.com/documentation/safari_release_notes/safari_12_1_release_notes?language=objc

Safari for iOS support VP8.

Seems to partially works will old licode (iPhone play video from other). Will try will last licode later

[mccob]

With last licode, get that error on Safari/iOS :

TypeError: c.setDirection is not a function. (In 'c.setDirection("sendonly")', 'c.setDirection' is undefined)
Maybe iOS provide VP8 for decoding but not for encoding (iPhone can see other partipant).

Someone get an idea ?

[mkhahani]

As per release notes "Safari 12.1 is included with iOS 12.2 and macOS 10.14.4."

@mccob
How did you get Safari 12.1? My iPhone has just received iOS 12.1.3 recently but not iOS 12.2.

[mccob]

As per release notes "Safari 12.1 is included with iOS 12.2 and macOS 10.14.4."

@mccob
How did you get Safari 12.1? My iPhone has just received iOS 12.1.3 recently but not iOS 12.2.

ah yes ! iOS beta program (open to everyone)

[mccob]

• it's now working with last iOS beta ! youpi !

• only had to comment lines that setDirection for mic and camera

• so only have to solve this issue (in safari stack ?) and this issue can be closed after

[mkhahani]

iOS 12.2 has been released finally! Any plan to support Safari officially?

[kekkokk]

can confirm vp8 implementation works flawlessly and simulcast too if you use the same chromestable stack

[mkhahani]

In my test the subscription from Safari fails unless the subscriber is capturing. Indeed the ICE connection is not completed before call to getUserMedia() and granting access to the browser in the subscriber side. Tested with both Licode v6 and v7 stable.

[mkhahani]

@kekkokk @mccob
Do you confirm this? If no then please let me know what version are you using exactly.

[kekkokk]

@mkhahani I'm really sorry but our version of Licode was branched from the v4 pre single-pc implementation and in these months, since our needs were not in line with the needs of the Licode authors (absolutely reasonable since our needs are very differents), we made ton of changes and rewrote the complete client sdp and libnice stack, so I really don't know how to help. I'll suggest you to take some hints with the EDGE support PR I made moths ago

[mkhahani]

@kekkokk You've made the right decision. The project dev line differs for many of us nowadays.

@jcague @lodoyun @aalonsog
It's years we've been waiting for the support for iOS and now that Apple has taken a big step toward WebRTC by supporting vp8 + simulcast, we are very close to it. Please give it a try and let this happen or give an advice at least. Thanks.

[AbhinandanAppsMaven]

Hi Dear, I am just implementing the your Demo following from your Github Repo in my native app but it is outdated please guide me for this. what if i implement this inside WebView can you please give me some rough scenario for this please. Thanks in Advance

[aztack]

Any ideas on setDirection() is undefined ?
Can I just replace setDirection("sendonly") with transceiver.direction = "sendonly"?

[RArtieda]

Please I need Help:
My WebRTC code works fine over Chrome but It doesnt work in Safari
I have this code script.js

var ICE_config= {
'iceServers': [
{
'urls': 'stun:stun.l.google.com:19302'
},
{
'urls': 'turn:192.158.29.39:3478?transport=udp',
'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA=',
'username': '28224511:1379330808'
}
]
}

function onSuccess() {};
function onError(error) {
console.error(error);
};

drone.on('open', error => {
if (error) {
return console.error(error);
}
room = drone.subscribe(roomName);
room.on('open', error => {
if (error) {
onError(error);
}
});
// We're connected to the room and received an array of 'members'
// connected to the room (including us). Signaling server is ready.
room.on('members', members => {
console.log('MEMBERS', members);
// If we are the second user to connect to the room we will be creating the offer
const isOfferer = members.length === 2;

var browser=get_browser();

// alert(browser.name);
//alert(browser.version);

if(browser.name=="Safari")
startWebRTCIOS(isOfferer);
else
startWebRTC(isOfferer);

});
});

// Send signaling data via Scaledrone
function sendMessage(message) {
drone.publish({
room: roomName,
message
});
}

function startWebRTCIOS(isOfferer) {
navigator.mediaDevices.getUserMedia({
audio: true,
video: true,
});
}

function startWebRTC(isOfferer) {
pc = new RTCPeerConnection(ICE_config);
pc.onicecandidate = event => {
if (event.candidate) {
sendMessage({'candidate': event.candidate});
}
};

// If user is offerer let the 'negotiationneeded' event create the offer
if (isOfferer) {
pc.onnegotiationneeded = () => {
pc.createOffer().then(localDescCreated).catch(onError);
}
}

// When a remote stream arrives display it in the #remoteVideo element
pc.ontrack = event => {
const stream = event.streams[0];
if (!remoteVideo.srcObject || remoteVideo.srcObject.id !== stream.id) {
remoteVideo.srcObject = stream;
}
};

navigator.mediaDevices.getUserMedia({
audio: true,
video: true,
}).then(stream => {
// Display your local video in #localVideo element
localVideo.srcObject = stream;
// Add your stream to be sent to the conneting peer
stream.getTracks().forEach(track => pc.addTrack(track, stream));
}, onError);

// Listen to signaling data from Scaledrone
room.on('data', (message, client) => {
// Message was sent by us
if (client.id === drone.clientId) {
return;
}

if (message.sdp) {
  // This is called after receiving an offer or answer from another peer
  pc.setRemoteDescription(new RTCSessionDescription(message.sdp), () => {
    // When receiving an offer lets answer it
    if (pc.remoteDescription.type === 'offer') {
      pc.createAnswer().then(localDescCreated).catch(onError);
    }
  }, onError);
} else if (message.candidate) {
  // Add the new ICE candidate to our connections remote description
  pc.addIceCandidate(
    new RTCIceCandidate(message.candidate), onSuccess, onError
  );
}

});

}

function localDescCreated(desc) {
pc.setLocalDescription(
desc,
() => sendMessage({'sdp': pc.localDescription}),
onError
);
}

function get_browser() {
var ua=navigator.userAgent,tem,M=ua.match(/(opera|chrome|safari|firefox|msie|trident(?=/))/?\s*(\d+)/i) || [];
if(/trident/i.test(M[1])){
tem=/\brv[ :]+(\d+)/g.exec(ua) || [];
return {name:'IE',version:(tem[1]||'')};
}
if(M[1]==='Chrome'){
tem=ua.match(/\bOPR|Edge/(\d+)/)
if(tem!=null) {return {name:'Opera', version:tem[1]};}
}
M=M[2]? [M[1], M[2]]: [navigator.appName, navigator.appVersion, '-?'];
if((tem=ua.match(/version/(\d+)/i))!=null) {M.splice(1,1,tem[1]);}
return {
name: M[0],
version: M[1]
};
}