New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Apple Safari #915

Open
mccob opened this Issue Jun 8, 2017 · 60 comments

Comments

Projects
None yet
@mccob
Copy link

mccob commented Jun 8, 2017

Apple Safari/Webkit support is here since Safari 11.

Apple use Google/WebRTC libwebrtc.

See :
https://webkit.org/blog/7726/announcing-webrtc-and-media-capture/
https://webkit.org/blog/7627/safari-technology-preview-32/

Available at least on Mac OS.

Safari tech preview is available for actual Mac OS Sierra.

Licode isn't working.

Seems a specific stack is needed.

First error solved, has to use window.RTCPeerConnection

WebkitRTCPeerConnection = window.RTCPeerConnection,

Other errors that must be solved :

  • [Error] Unhandled Promise Rejection: TypeError: Type error createObjectURL
  • [Error] TypeError: Error creating RTCPeerConnection initializeRTCPeerConnection
  • [Error] TypeError: undefined is not an object (evaluating 'stream.pc.processSignalingMessage')
    (anonymous function)

if you want to help :)

@mccob mccob changed the title Support Safari Support Apple Safari Jun 8, 2017

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Jun 9, 2017

I'll probably look into it next week! have a nice day!

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Jun 9, 2017

Finally!!
It's a good time for migrating to adapter.js and get rid of all these browser stacks.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Jun 9, 2017

We might discuss that, is not that straight forward. we have always to differentiate by chrome and firefox for little things (ex. opus nack, the "a=" in front of ices, the filtering out of the end candidate in chrome etc.) so maybe we'll always have the various stacks.
but with a refactor of the sdp parser in C++ we might have good results with adapter.

@jcague

This comment has been minimized.

Copy link
Contributor

jcague commented Jun 9, 2017

I've been trying this week. I was able to create a videoconference with publishers sending video/audio from Chrome, Firefox and Safari, after solving all the errors mentioned by @mccob. Things to consider when we implement it:

  • Safari only supports h.264.
  • h.264 support in Licode is not perfect, we might need to add better support for fmtp lines in the SDP. I found some issue when sending H.264 from Firefox because of that, but I solved it with a quick and dirty hack.
  • I was able to send video from Safari, and I received and rendered it in Chrome and Firefox.
  • I didn't receive any data in Safari from publishers in Chrome and Firefox. The API is slightly different and I don't know if things like OfferToReceiveVideo or Transceivers are completely implemented in Safari's current implementation (WebRTC in Safari is still a WIP, but really close to be completed I think, and they're probably thinking of adding native support for WebRTC in iOS 11, I think I saw something about it in their last keynote).

Completely agree with @kekkokk, with might need to discuss adapterjs and/or other solutions. Anyway, thanks @mccob for playing with Safari. Can you try enabling h264 in the rtp_media_config?

@jcague

This comment has been minimized.

Copy link
Contributor

jcague commented Jun 9, 2017

FYI: master...jcague:poc/safari this is the result of a couple of hours trying to make it work... I'm not proud of it, it's plenty of hacks!! so please don't use it in production, staging or even debugging 😂

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Jun 9, 2017

@jcague any plan to try edge? I opened the most detailed post i could @ DISCOURSE but I had no response :(
in business environments is way more used than safari.
Another question I never asked: is licode h264 ready?
I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264.
Do you think I could switch seamlessly into h264 without problems? or something (ex. externaloutput) will break? I never tried....

@menelike

This comment has been minimized.

Copy link

menelike commented Jun 14, 2017

@jcague

h.264 support in Licode is not perfect

Are you talking about P2P oder the MCU mode?

Asking because we need h264 support on the MCU, because of

I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now* and observed performance problems, we'd love to try h264 since this might be a hardware support related issue.

*We use adapter.js and actually the chrome stack for iOS, works fine for a long time now (but we don't use the latest features like SimulCast, but Opus, Isac etc works fine!)

@thehappycoder

This comment has been minimized.

Copy link

thehappycoder commented Sep 8, 2017

Is there any ETA on when Licode starts supporting Safari? Even if it's not perfect but good-enough for some limited use cases.

@miniruwan

This comment has been minimized.

Copy link

miniruwan commented Sep 18, 2017

Calls

I found the reason why safari doesn't receive media from other browsers.

I didn't receive any data in Safari from publishers in Chrome and Firefox.

MCU doesn't send answer sdp until it receives ICE candidates. But safari doesn't generate and send ICE candidates. https://bugs.webkit.org/show_bug.cgi?id=176157
One way to fix this is a shim in adapter.js (https://github.com/webrtc/adapter/issues/661)

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

Recording

For call recording to work, ExternalOutput probably needs to be changed to follow the h264 specs. Is there any ETA for this?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 18, 2018

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

@miniruwan Could you get it to work with Licode? I built EC with the latest adapter.js (v6.3.2) but still could not receive audio/video in Safari.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Aug 27, 2018

try with this SafariStack

/* global */

import Logger from '../utils/Logger';
import BaseStack from './BaseStack';

const SafariStack = (specInput) => {
  Logger.info('Starting Safari stack');
  const spec = specInput;

  const newIces = [];

  spec.iceServers.forEach((server) => {
    const newServer = {};
    if (server.url) {
      newServer.urls = server.url;
    }
    if (server.username) {
      newServer.username = server.username;
    }
    if (server.credential) {
      newServer.credential = server.credential;
    }
    newIces.push(newServer);
  });

  spec.iceServers = newIces;

  const that = BaseStack(spec);

  return that;
};

export default SafariStack;
@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 28, 2018

Thanks for the response but it didn't help :(

Hopefully in p2p mode, Chrome2Safari and Safari2Chrome work well on both MacOS and iOS. I believe it can be fixed to work also in MCU mode.

Even I think Chrome over iOS must be able to receive audio/video after Safari added support for WebRTC since Chrome uses WKWebView component for iOS and it supports RTCPeerConnection now.

@menelike

This comment has been minimized.

Copy link

menelike commented Aug 28, 2018

added support for WebRTC

Just my two cents...

Well yes, they "support" webRTC but still, VP8 is not supported just to name one of the many drawbacks. Without going into the details on how flawed webRTC on iOS is, as long as you have to rely on mobile browsers between iOS and Android, results are unpredictable. Your best chance will be to stick to h264, but then licode doesn't support h264 within the MCU.

IMHO for a complete working support for all devices including desktop browsers, iOS and Android clients need to be implemented natively. In addition/or instead a transcoding MCU is advised.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Aug 28, 2018

actually Licode does support for h264. and works well with safari. I actually made lots of changes to be able to, mainly you can see my EDGE stack PR and add the stack I posted few comments above

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 28, 2018

@menelike
Thanks for your advice. I'm just trying to find a way to add support for iOS in my app and it's become a nightmare.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now

Actually I was going to write my own hybrid app using cordova+iosrtc plugin, as discussed here, but I figure it out that the provided iOSStack is out of sync with the latest licode changes. Do you have any update on that?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 28, 2018

@kekkokk
Are you saying you could get the mcu to work with Safari+iOS? Then where can I find your PR?

@menelike

This comment has been minimized.

Copy link

menelike commented Aug 29, 2018

@mkhahani
We have to use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc as well, and it's a pain. The project is almost dead, one major problem will be, that h264 playback will result in a green screen (h264 is the first choice on iOS because it should reduce the CPU load)

I think all work regarding webRTC has shifted from Cordova to react native (https://github.com/oney/react-native-webrtc), imho Cordova and webRTC will never work, especially not since getUserMedia is not available in WKWebView.

We also switched from licode to https://github.com/meetecho/janus-gateway, but this won't help you in your situation.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Aug 30, 2018

@mkhahani We actually have one dev only for iOS and we did a native app with the webrtc core.
the webrtc core supports vp8 on iphone.

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 30, 2018

@menelike

imho Cordova and webRTC will never work

But it was working in my tests before the latest changes in Erizo Client.

We also switched from licode to https://github.com/meetecho/janus-gateway

Do you really found Janus better than Licode?!

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 30, 2018

@kekkokk

We actually have one dev only for iOS and we did a native app with the webrtc core.

Wow that's a lot of work! What do you mean by the "webrtc core" exactly? Are you using the one provided by webrtc.org? And is that compatible with Licode?

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Aug 30, 2018

yeah sure! that's a ton of works but, actually, is the one used by chrome. so the sdps are quite similar and supports vp8.

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 31, 2018

@kekkokk
@menelike
What do you think about Temasys WebRTC Plugin which is used by Jitsi to add support for WebRTC in IE and Safari?

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Aug 31, 2018

we are using this only for IE and works quite well (produces sdp very similar to chrome).
For the safari side, we found very difficult, now that safari 11 supports natively webrtc, to interoperate with the plugin.
It was too difficult to differentiate safari versions and choose to use the plugin or not.
ATM we enabled only safari >= 11 in our platform so the call will init in h264 room

ps. doesn't support simulcast ofc

@zevarito

This comment has been minimized.

Copy link
Contributor

zevarito commented Aug 31, 2018

@mkhahani There is an IOS ObjC client for Licode I've done a while ago, not sure how it works with latest version of Licode since I didn't had chance to work on it lately but I am sure if it doesn't work out-of-the-box it just need some signaling-protocol update almost. PR and maintainers are welcome, Link bellow in case you want to check it out.

https://github.com/zevarito/Licode-ErizoClientIOS

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Aug 31, 2018

Thank you @zevarito, I am aware of your iOS app but I'm trying to avoid writing native since I've done a lot of work on other parts of my app like ui, signaling, etc.

@jaydgoss

This comment has been minimized.

Copy link

jaydgoss commented Sep 3, 2018

@mkhahani Did you find that Chrome to Safari, iOS to macOS combinations work in p2p mode?

@jcague

This comment has been minimized.

Copy link
Contributor

jcague commented Sep 13, 2018

@mkhahani we're currently adding playsinline within the VideoPlayer so it should work, did you need to add any other attribute?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Sep 13, 2018

@jcague Very good! No I don't think so.

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Oct 12, 2018

We decided to develop native app as our iOS solution. Some friends suggested React Native but Google Flutter seems also a good choice as a cross platform SDK. My concern is just about how does it deal with WebRTC. Since we may share part of the code publicly and it suppose to work with Licode, I would like to have your thoughts and ideas.

@pooyarm

This comment has been minimized.

Copy link

pooyarm commented Oct 12, 2018

We decided to develop native app as our iOS solution. Some friends suggested React Native but Google Flutter seems also a good choice as a cross platform SDK. My concern is just about how does it deal with WebRTC. Since we may share part of the code publicly and it suppose to work with Licode, I would like to have your thoughts and ideas.

@mkhahani If you want to use React Native there is a webrtc library which can be used and licode js client can talk with that.
https://github.com/oney/react-native-webrtc
I used these two to build conference on react native and this worked with some adjustments.
For instance, to have user's stream with Erizo.Stream, first you need to use getUserMedia method from 'react-native-webrtc' library and then create Erizo.Stream object using created stream, here is the code example:

const WebrtcGetUserMedia = (isFront = true, attributes = {}) => {
    return new Promise((success, error) => {
        MediaStreamTrack
        .getSources()
        .then(sourceInfos => {
            let videoSourceId;
            for (let i = 0; i < sourceInfos.length; i++) {
                const sourceInfo = sourceInfos[i];
                if(sourceInfo.kind == "video" && sourceInfo.facing == (isFront ? "front" : "back")) {
                    videoSourceId = sourceInfo.id;
                }
            }
            return getUserMedia({
                audio: true,
                video: {
                    mandatory: {
                        minFrameRate: 30
                    },
                    facingMode: (isFront ? "user" : "environment"),
                    optional: (videoSourceId ? [{sourceId: videoSourceId}] : [])
                }
            });
        })
        .then(stream => {
            var streamObj = Erizo.Stream(undefined, {
                stream,
                video: true,
                audio: true,
                local: true,
                data: true,
                attributes
            });
            success(streamObj);
        })
        .catch((e) => {
            error(e);
        });
    });
}
@mccob

This comment has been minimized.

Copy link
Author

mccob commented Oct 12, 2018

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Oct 12, 2018

Have you seen that VP8 support will be in Safari soon ?
Available in Safari Preview Release

I can't believe that Apple has added support for VP8!! It's the power of WebRTC 😎. So better to wait and see what happens.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Oct 12, 2018

This is a great news for the community! I talked with fippo and seems that VP8 in the developer preview is in a stage that features can't be discoverable, so we should wait a few more weeks to better understand the behaviour, but I mean, it's only matter of time. I never thought Apple would have release VP8. I'll try to give it a try next week!

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Oct 16, 2018

I have tested with adapter.js and without (with a version of a licode built a few months ago). I have javascript errors. Maybe javascript client stack need some update (or maybe it's already updated on last revision).

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Oct 23, 2018

I tested safari vp8 with our current modified webrtc stack in licode and works like a charm. reacts very well to nack and bandwidth drop. way more stable than current Edge implementation.

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Oct 24, 2018

@kekkokk
Interesting!! May I have your stack please?

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Oct 25, 2018

Have you tried with your basestack? it should be work fine. I mean, my base stack (js) is very similar to the project one. If it does not work I suggest you to try to disable the single peerconnection feature and comment this line:

if (evt.stream.id === that.getLabel()) {

If it does not work it's maybe something we changed in c++ side but I don't think

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Oct 27, 2018

Does it mean I need to change the following code to use BaseStack?

} else if (this.browser === 'safari') {
Logger.debug('Safari using Chrome Stable Stack');
this.stack = ChromeStableStack(spec);

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Oct 27, 2018

no chrome stack should be ok, it only shim the simulcast

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Oct 27, 2018

I get the following errors when publishing/subscribing with Safari using VP8 while it works fine with H264:

image

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Oct 31, 2018

update :

working without any update with last licode release (and Safari Tech Preview). Thank you @kekkokk and everyone. Hope Apple will provide this to iOS soon.

@lidedongsn

This comment has been minimized.

Copy link
Contributor

lidedongsn commented Nov 6, 2018

publish from Safari ,sdp "a=sendrecv", Safari send all candidates, and licode acts "All candidates received", but there is not to continue, nothing going on! ICE state not ready. why?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Nov 22, 2018

@mccob
iOS 12 is out and it works using Safari in P2P mode! But I'm still getting the above error in MCU mode. What version of Licode are you using? Any customization?

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Nov 22, 2018

No relation with VP8, iOS 12 was out before Apple announce VP8 support.

Licode can work with h264, in P2P mode only ? Because I already try in no P2P mode it's not working.

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Nov 22, 2018

I see but I also get the error on Safari Tech Preview. How could you get it to work?

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Nov 22, 2018

Do you enable vp8 support on preview options ?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Nov 22, 2018

Ah! No I never knew about that.

@kekkokk

This comment has been minimized.

Copy link
Contributor

kekkokk commented Nov 23, 2018

screen shot 2018-11-23 at 10 38 05
screen shot 2018-11-23 at 10 38 16

Great job Apple! once in a while

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Nov 24, 2018

Yes it's working now. 👍
Thank you.

@lidedongsn

This comment has been minimized.

Copy link
Contributor

lidedongsn commented Dec 6, 2018

safari only subscribe,no video play!
MediaElementSession::autoplayPermitted(112EA802) Returning FALSE because element is not visible in the viewport

@pawanvivekananda

This comment has been minimized.

Copy link

pawanvivekananda commented Dec 13, 2018

screenshot 2018-12-13 at 12 18 36 am

I am facing same issue in safari 12 on MAC

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Feb 4, 2019

https://developer.apple.com/documentation/safari_release_notes/safari_12_1_release_notes?language=objc

Safari for iOS support VP8.

Seems to partially works will old licode (iPhone play video from other). Will try will last licode later

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Feb 5, 2019

With last licode, get that error on Safari/iOS :

TypeError: c.setDirection is not a function. (In 'c.setDirection("sendonly")', 'c.setDirection' is undefined)
Maybe iOS provide VP8 for decoding but not for encoding (iPhone can see other partipant).

Someone get an idea ?

@mkhahani

This comment has been minimized.

Copy link
Contributor

mkhahani commented Feb 5, 2019

As per release notes "Safari 12.1 is included with iOS 12.2 and macOS 10.14.4."

@mccob
How did you get Safari 12.1? My iPhone has just received iOS 12.1.3 recently but not iOS 12.2.

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Feb 5, 2019

As per release notes "Safari 12.1 is included with iOS 12.2 and macOS 10.14.4."

@mccob
How did you get Safari 12.1? My iPhone has just received iOS 12.1.3 recently but not iOS 12.2.

ah yes ! iOS beta program (open to everyone)

@mccob

This comment has been minimized.

Copy link
Author

mccob commented Feb 6, 2019

  • it's now working with last iOS beta ! youpi !

  • only had to comment lines that setDirection for mic and camera

  • so only have to solve this issue (in safari stack ?) and this issue can be closed after

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment