Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS 11 Safari Needs Additional Properties #929

Closed
kylemcdonald opened this issue Sep 19, 2017 · 43 comments
Closed

iOS 11 Safari Needs Additional Properties #929

kylemcdonald opened this issue Sep 19, 2017 · 43 comments

Comments

@kylemcdonald
Copy link

Browsers and versions affected

iOS 11 Safari on iPhone (not on iPad)

Description

getUserMedia() does not start unless some additional properties are added to the <video> tag: playsinline and autoplay, and controls="true". Details are being tracked in this thread. It looks like once you run an example with controls="true" once, then it's possible to remove the controls for that page in the future. A modified working example is here.

Steps to reproduce

Open the gum sample on an iPhone 6S running iOS 11 Safari.

Expected results

Shows video.

Actual results

Video is black with no errors.

@fippo
Copy link
Collaborator

fippo commented Sep 20, 2017

@youennf you might want to take a look

@tkbremnes
Copy link

tkbremnes commented Sep 20, 2017

I've verified that it works. How we at appear.in solved it:

// Hacks for Mobile Safari
video.setAttribute("playsinline", true);
video.setAttribute("controls", true);
setTimeout(() => {
    video.removeAttribute("controls");
});

Update: you don't need the controls=true hack above, just the playsinline.

@youennf
Copy link
Contributor

youennf commented Sep 20, 2017

Use of playsinline is currently required on iOS to actually play any video inline.
There is no special getUserMedia/webrtc treatment here at the moment.

The need for the controls="true" trick is unexpected so there is probably a bug here.
tkbremnes, are you confirming that the "controls" trick was needed for you?
You need to do that for every page load, every video element?

@tkbremnes
Copy link

I'll have a look and see if the controls=true is actually needed.

@tkbremnes
Copy link

It appears that the controls=true thing wasn't needed. Thanks :)

@benbro
Copy link

benbro commented Sep 21, 2017

In my tests controls="true" is needed.
Without it remote stream doesn't play.

@youennf
Copy link
Contributor

youennf commented Sep 21, 2017

See also related #926

@seeingwithsound
Copy link

I have a WebRTC app that works in Firefox and Chrome on Android and Windows, but user reports tell me that it still fails in Safari 11 on iOS 11. What could I be missing? I've added the playsinline to The vOICe Web App https://www.seeingwithsound.com/webvoice.htm

Thanks!

@youennf
Copy link
Contributor

youennf commented Sep 22, 2017

Difficult to say.
Loading your web site on Safari, I can see that two NotSupported exceptions are raised in your web audio code, when trying to call createBuffer on a webKitAudioContext.
Is that the issue you are talking about?
If so, it probably means that some of the parameter values of the sample rate or the number of channels might not supported.

@seeingwithsound
Copy link

It might be. Blind iPhone users told me that they got no sound whatsoever, so the exception would explain that. Unfortunately I have no iPhone or iPad so am for the moment working with educated guesses. I'm using 2-channel stereo with a 16000 Hz sample rate, which is a very common format (AFAIK Web Audio does not support querying supported formats https://webaudio.github.io/web-audio-api/). Thank you for trying, because it gives me a useful hint in what direction to look. I'll Google around some more. Perhaps it is as trivial as using a sample rate of 22050 instead of 16000. The former link states "An implementation MUST support sample rates in at least the range 8000 to 96000", but an older version stated "An implementation must support sample-rates in at least the range 22050 to 96000" https://groups.google.com/a/chromium.org/forum/#!topic/blink-reviews/a_OfM-a5ztI so maybe Apple used the older spec which could exclude 16000?

@youennf
Copy link
Contributor

youennf commented Sep 22, 2017

Is is probably not the right place, might be better to file a bug in bugs.webkit.org.
WebKit is implementing an older WebAudio spec version so 16000 is probably unsupported. 22050 should.

@youennf
Copy link
Contributor

youennf commented Sep 22, 2017

Note also that iPhone/iPads might require a user gesture to produce sound.

@seeingwithsound
Copy link

Thanks a lot! I've modified my web app to use a sample rate of 22050 instead of 16000 to see if that makes a difference https://www.seeingwithsound.com/webvoice.htm (this might require a browser cache refresh). Yes, I am aware of the annoying gesture requirement and have advised blind testers to try press the app's Reset button to get things going (which by itself ought to sound a 2-tone beep). If the main issue is indeed the 22050 vs 16000 then I'll file a bug report about it (the 16000 was never a problem in Firefox, IceWeasel, Chrome and Edge).

@youennf
Copy link
Contributor

youennf commented Sep 22, 2017

Loaded and I now get a gum prompt and see b/w frames...

@seeingwithsound
Copy link

Cool, that is what is supposed to happen. Getting any sounds associated with the B&W camera frames?

@youennf
Copy link
Contributor

youennf commented Sep 23, 2017

Getting some sound on Safari Mac

@seeingwithsound
Copy link

seeingwithsound commented Sep 23, 2017

Fabulous! Over the past 12 hours I received multiple reports from blind users with Safari under iOS 11 on iPhone 7 and iPhone 6S that it worked for them too, provided they hit the Play button at startup to take care of Apple's gesture requirement. They got the live soundscapes that they were already used to hearing on other platforms. Your remarks concerning NotSupported with createBuffer proved invaluable in finding the root cause of the problems. Thank you.

@youennf
Copy link
Contributor

youennf commented Sep 28, 2017

https://webrtc.github.io/samples/src/content/getusermedia/gum/ was updated to use playsinline.
Let me know if this example does not work out of the box now.

@anton-bot
Copy link

If anyone is having issues with iOS 11 just like I did, it turned out that iOS11 seems to force the sampling rate to 48000, even though I specifically requested 16000.

@mrwizzer
Copy link

This is very promising! Can anyone direct me to an example of a simple audio recorder using getUserMedia working successfully on iOS?

@anton-bot
Copy link

@mrwizzer Microsoft claims that their Bing Speech SDK will work, but I haven't tested.

@mrwizzer
Copy link

Thanks @catcher-in-the-try, but that looks more like a speech to text API. While it looks very cool, I'm just looking for a very simple way to implement a voice recorder within the browser to record a short audio snippet and play it back immediately. No need even to download the resulting audio file.

Can anyone direct me to a tutorial or sample of something that simple using getUserMedia or similar?

@aiquantong
Copy link

HI, there are demo, https://webrtc.github.io/samples/. it is very good adapter in the popular browser.

@mrwizzer
Copy link

Aiquantong, thanks, this looks like a great comprehensive list.

But the "Record Stream" demo doesn't work on iOS. They're using the MediaRecorder API, which isn't supported on iOS.

Does anyone know if recording the audio stream using WebRTC is possible yet on iOS?

@aiquantong
Copy link

Hi mrwizzer. It is more critical issue to make the webRTC audio not to work. Now, it is blocking my job, very badly.
screen shot 2017-12-01 at 4 55 34 pm

@mrwizzer
Copy link

mrwizzer commented Dec 4, 2017

This is the closest example of successful audio recording I've found, but still not 100% there. It works solidly in Safari on MacOS, as well as Chrome and Firefox, Android too! And, sometimes on iOS. About 50% of the time I get a successful recording. The other half of the time it shows "Error" with no recording.

Here's the example:
https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html

Anyone have better luck than I am getting a successful audio recording on iOS?

@ee11131
Copy link

ee11131 commented Jan 12, 2018

Is there a fix for this yet? In my case I have no sound out o Safari.

@youennf
Copy link
Contributor

youennf commented Jan 12, 2018

@ee11131, it is not clear what you are requesting.
If you have a specific request, can you file a bug at bugs.webkit.org and cc me?

@ee11131
Copy link

ee11131 commented Jan 12, 2018

https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html is not working for me in Safari 11, I have a microphone connected with USB. No errors are shown, it simply records nothing at all.

@youennf
Copy link
Contributor

youennf commented Jan 12, 2018

Which API is it using to record audio? Is it web audio? MediaRecorder API?
MediaRecorder is not supported in Safari 11.

@vbeattie
Copy link

I am having similar issues as reported above for audio recording. For me, https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html works occasionally (I would say far less than 50% of the time). When it does not work, there is no error, but the audio that is being returned/recorded is just a stream of zeroes.

I am currently using the developer iOS beta 11.2.5 b7

We have a similar experience with our web app which records audio and which works on other browser/OS combinations. It will occasionally work, but mostly returns silence.

Is there some trick similar to what is mentioned for video above that is needed to get this to work? Or some way to reset the audio interface so that it works every time?

@elbalkini
Copy link

@vbeattie have you tried the new IOS 11.3? any luck in this version?

@vbeattie
Copy link

I am using 11.3 it now - seems somewhat better(?), but hard to say. Problem is definitely not resolved as I still get the muted/audio is stream of zeroes behavior sometimes.

@KaptenJansson
Copy link
Contributor

This bug has become unactionable, please file a new one or better yet a PR with the changes you need.

@tetreault
Copy link

demo in OP comment not working in iOS 11.4 on iphone X :(

@youennf
Copy link
Contributor

youennf commented Jun 5, 2018

@tetreault, could you please file a bug in bugs.webkit.org?

@tetreault
Copy link

you can disregard @youennf -- was a weird mistake on my end :)

@alienpavlov
Copy link

@tetreault what kind of mistake it was?.. I'm having similar issue

@tetreault
Copy link

I updated my version of iOS @alienpavlov and when i did, for some reason, the javascript was disabled in my safari settings. No clue how that happened.

@tetreault
Copy link

@alienpavlov here's my full JS demo if you need it:

"use strict";

document.addEventListener("DOMContentLoaded", () => {
  const startButton = document.getElementById("start-video");
  const frontCamVideo = document.getElementById("front-camera");

  let constraints = (window.constraints = {
    audio: false,
    video: {
      facingMode: "user"
    }
  });

  function handleSuccess(stream) {
    var videoTracks = stream.getVideoTracks();
    console.log("Got stream with constraints:", constraints);
    console.log("Using video device: " + videoTracks[0].label);
    stream.oninactive = function() {
      console.log("Stream inactive");
    };
    window.stream = stream; // make variable available to browser console
    frontCamVideo.srcObject = stream;
  }

  function handleError(error) {
    if (error.name === "ConstraintNotSatisfiedError") {
      errorMsg(
        "The resolution " +
          constraints.video.width.exact +
          "x" +
          constraints.video.width.exact +
          " px is not supported by your device."
      );
    } else if (error.name === "PermissionDeniedError") {
      errorMsg(
        "Permissions have not been granted to use your camera and " +
          "microphone, you need to allow the page access to your devices in " +
          "order for the demo to work."
      );
    }
    errorMsg("getUserMedia error: " + error.name, error);
  }

  function errorMsg(msg, error) {
    console.error(msg);
    if (typeof error !== "undefined") {
      console.error(error);
    }
  }

  startButton.addEventListener("click", e => {
    navigator.mediaDevices
      .getUserMedia(constraints)
      .then(handleSuccess)
      .catch(handleError);
  });
});

@PrItHvI7
Copy link

Does Progressive Web Apps on iOS support “getUserMedia/Stream API” ? i am developing an application using Quagga barcode scanner, where it works fine on ios safari but while accessing through webApp it crashes with error below. [Error] ERROR — Error: Uncaught (in promise): TypeError: undefined is not an object (evaluating ‘k.inputStream.type’)

@tetreault
Copy link

no @PrltHvl7, i hate to break it to you but apple either intentionally or unintentionally did not integrate webrtc into anything that uses UIWebview or WKWebview. It really sucks, i had to split up my web app so mobile safari and android use webcam and everything else on iphone (ios chrome/ff/social in-app browsers/stand-alone PWA) utilized a photo upload since those webviews don’t have webrtc support

@webrtc webrtc locked as resolved and limited conversation to collaborators Aug 31, 2018
@fippo
Copy link
Collaborator

fippo commented Aug 31, 2018

Locking this conversation which has been very useful and became the de-facto reference for this :-)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests