New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS 11 Safari Needs Additional Properties #929

Closed
kylemcdonald opened this Issue Sep 19, 2017 · 43 comments

Comments

Projects
None yet
@kylemcdonald

kylemcdonald commented Sep 19, 2017

Browsers and versions affected

iOS 11 Safari on iPhone (not on iPad)

Description

getUserMedia() does not start unless some additional properties are added to the <video> tag: playsinline and autoplay, and controls="true". Details are being tracked in this thread. It looks like once you run an example with controls="true" once, then it's possible to remove the controls for that page in the future. A modified working example is here.

Steps to reproduce

Open the gum sample on an iPhone 6S running iOS 11 Safari.

Expected results

Shows video.

Actual results

Video is black with no errors.

@fippo

This comment has been minimized.

Contributor

fippo commented Sep 20, 2017

@youennf you might want to take a look

@tkbremnes

This comment has been minimized.

tkbremnes commented Sep 20, 2017

I've verified that it works. How we at appear.in solved it:

// Hacks for Mobile Safari
video.setAttribute("playsinline", true);
video.setAttribute("controls", true);
setTimeout(() => {
    video.removeAttribute("controls");
});

Update: you don't need the controls=true hack above, just the playsinline.

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 20, 2017

Use of playsinline is currently required on iOS to actually play any video inline.
There is no special getUserMedia/webrtc treatment here at the moment.

The need for the controls="true" trick is unexpected so there is probably a bug here.
tkbremnes, are you confirming that the "controls" trick was needed for you?
You need to do that for every page load, every video element?

@tkbremnes

This comment has been minimized.

tkbremnes commented Sep 21, 2017

I'll have a look and see if the controls=true is actually needed.

@tkbremnes

This comment has been minimized.

tkbremnes commented Sep 21, 2017

It appears that the controls=true thing wasn't needed. Thanks :)

@benbro

This comment has been minimized.

benbro commented Sep 21, 2017

In my tests controls="true" is needed.
Without it remote stream doesn't play.

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 21, 2017

See also related #926

@seeingwithsound

This comment has been minimized.

seeingwithsound commented Sep 22, 2017

I have a WebRTC app that works in Firefox and Chrome on Android and Windows, but user reports tell me that it still fails in Safari 11 on iOS 11. What could I be missing? I've added the playsinline to The vOICe Web App https://www.seeingwithsound.com/webvoice.htm

Thanks!

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 22, 2017

Difficult to say.
Loading your web site on Safari, I can see that two NotSupported exceptions are raised in your web audio code, when trying to call createBuffer on a webKitAudioContext.
Is that the issue you are talking about?
If so, it probably means that some of the parameter values of the sample rate or the number of channels might not supported.

@seeingwithsound

This comment has been minimized.

seeingwithsound commented Sep 22, 2017

It might be. Blind iPhone users told me that they got no sound whatsoever, so the exception would explain that. Unfortunately I have no iPhone or iPad so am for the moment working with educated guesses. I'm using 2-channel stereo with a 16000 Hz sample rate, which is a very common format (AFAIK Web Audio does not support querying supported formats https://webaudio.github.io/web-audio-api/). Thank you for trying, because it gives me a useful hint in what direction to look. I'll Google around some more. Perhaps it is as trivial as using a sample rate of 22050 instead of 16000. The former link states "An implementation MUST support sample rates in at least the range 8000 to 96000", but an older version stated "An implementation must support sample-rates in at least the range 22050 to 96000" https://groups.google.com/a/chromium.org/forum/#!topic/blink-reviews/a_OfM-a5ztI so maybe Apple used the older spec which could exclude 16000?

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 22, 2017

Is is probably not the right place, might be better to file a bug in bugs.webkit.org.
WebKit is implementing an older WebAudio spec version so 16000 is probably unsupported. 22050 should.

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 22, 2017

Note also that iPhone/iPads might require a user gesture to produce sound.

@seeingwithsound

This comment has been minimized.

seeingwithsound commented Sep 22, 2017

Thanks a lot! I've modified my web app to use a sample rate of 22050 instead of 16000 to see if that makes a difference https://www.seeingwithsound.com/webvoice.htm (this might require a browser cache refresh). Yes, I am aware of the annoying gesture requirement and have advised blind testers to try press the app's Reset button to get things going (which by itself ought to sound a 2-tone beep). If the main issue is indeed the 22050 vs 16000 then I'll file a bug report about it (the 16000 was never a problem in Firefox, IceWeasel, Chrome and Edge).

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 22, 2017

Loaded and I now get a gum prompt and see b/w frames...

@seeingwithsound

This comment has been minimized.

seeingwithsound commented Sep 22, 2017

Cool, that is what is supposed to happen. Getting any sounds associated with the B&W camera frames?

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 23, 2017

Getting some sound on Safari Mac

@seeingwithsound

This comment has been minimized.

seeingwithsound commented Sep 23, 2017

Fabulous! Over the past 12 hours I received multiple reports from blind users with Safari under iOS 11 on iPhone 7 and iPhone 6S that it worked for them too, provided they hit the Play button at startup to take care of Apple's gesture requirement. They got the live soundscapes that they were already used to hearing on other platforms. Your remarks concerning NotSupported with createBuffer proved invaluable in finding the root cause of the problems. Thank you.

@youennf

This comment has been minimized.

Contributor

youennf commented Sep 28, 2017

https://webrtc.github.io/samples/src/content/getusermedia/gum/ was updated to use playsinline.
Let me know if this example does not work out of the box now.

@catcher-in-the-try

This comment has been minimized.

catcher-in-the-try commented Oct 26, 2017

If anyone is having issues with iOS 11 just like I did, it turned out that iOS11 seems to force the sampling rate to 48000, even though I specifically requested 16000.

@mrwizzer

This comment has been minimized.

mrwizzer commented Nov 27, 2017

This is very promising! Can anyone direct me to an example of a simple audio recorder using getUserMedia working successfully on iOS?

@catcher-in-the-try

This comment has been minimized.

catcher-in-the-try commented Nov 28, 2017

@mrwizzer Microsoft claims that their Bing Speech SDK will work, but I haven't tested.

@mrwizzer

This comment has been minimized.

mrwizzer commented Nov 28, 2017

Thanks @catcher-in-the-try, but that looks more like a speech to text API. While it looks very cool, I'm just looking for a very simple way to implement a voice recorder within the browser to record a short audio snippet and play it back immediately. No need even to download the resulting audio file.

Can anyone direct me to a tutorial or sample of something that simple using getUserMedia or similar?

@aiquantong

This comment has been minimized.

aiquantong commented Nov 29, 2017

HI, there are demo, https://webrtc.github.io/samples/. it is very good adapter in the popular browser.

@mrwizzer

This comment has been minimized.

mrwizzer commented Nov 29, 2017

Aiquantong, thanks, this looks like a great comprehensive list.

But the "Record Stream" demo doesn't work on iOS. They're using the MediaRecorder API, which isn't supported on iOS.

Does anyone know if recording the audio stream using WebRTC is possible yet on iOS?

@aiquantong

This comment has been minimized.

aiquantong commented Dec 1, 2017

Hi mrwizzer. It is more critical issue to make the webRTC audio not to work. Now, it is blocking my job, very badly.
screen shot 2017-12-01 at 4 55 34 pm

@mrwizzer

This comment has been minimized.

mrwizzer commented Dec 4, 2017

This is the closest example of successful audio recording I've found, but still not 100% there. It works solidly in Safari on MacOS, as well as Chrome and Firefox, Android too! And, sometimes on iOS. About 50% of the time I get a successful recording. The other half of the time it shows "Error" with no recording.

Here's the example:
https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html

Anyone have better luck than I am getting a successful audio recording on iOS?

@ee11131

This comment has been minimized.

ee11131 commented Jan 12, 2018

Is there a fix for this yet? In my case I have no sound out o Safari.

@youennf

This comment has been minimized.

Contributor

youennf commented Jan 12, 2018

@ee11131, it is not clear what you are requesting.
If you have a specific request, can you file a bug at bugs.webkit.org and cc me?

@ee11131

This comment has been minimized.

ee11131 commented Jan 12, 2018

https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html is not working for me in Safari 11, I have a microphone connected with USB. No errors are shown, it simply records nothing at all.

@youennf

This comment has been minimized.

Contributor

youennf commented Jan 12, 2018

Which API is it using to record audio? Is it web audio? MediaRecorder API?
MediaRecorder is not supported in Safari 11.

@vbeattie

This comment has been minimized.

vbeattie commented Jan 24, 2018

I am having similar issues as reported above for audio recording. For me, https://www.webrtc-experiment.com/RecordRTC/simple-demos/audio-recording.html works occasionally (I would say far less than 50% of the time). When it does not work, there is no error, but the audio that is being returned/recorded is just a stream of zeroes.

I am currently using the developer iOS beta 11.2.5 b7

We have a similar experience with our web app which records audio and which works on other browser/OS combinations. It will occasionally work, but mostly returns silence.

Is there some trick similar to what is mentioned for video above that is needed to get this to work? Or some way to reset the audio interface so that it works every time?

@elbalkini

This comment has been minimized.

elbalkini commented Jan 26, 2018

@vbeattie have you tried the new IOS 11.3? any luck in this version?

@vbeattie

This comment has been minimized.

vbeattie commented Jan 31, 2018

I am using 11.3 it now - seems somewhat better(?), but hard to say. Problem is definitely not resolved as I still get the muted/audio is stream of zeroes behavior sometimes.

PeterOrneholm added a commit to PeterOrneholm/instascan that referenced this issue Feb 3, 2018

Add playsinline to work on iOS11
The preview is black on iOS11 but playsinline should work according to:
webrtc/samples#929 (comment)
@KaptenJansson

This comment has been minimized.

Member

KaptenJansson commented Mar 8, 2018

This bug has become unactionable, please file a new one or better yet a PR with the changes you need.

@tetreault

This comment has been minimized.

tetreault commented Jun 4, 2018

demo in OP comment not working in iOS 11.4 on iphone X :(

@youennf

This comment has been minimized.

Contributor

youennf commented Jun 5, 2018

@tetreault, could you please file a bug in bugs.webkit.org?

@tetreault

This comment has been minimized.

tetreault commented Jun 5, 2018

you can disregard @youennf -- was a weird mistake on my end :)

@alienpavlov

This comment has been minimized.

alienpavlov commented Jun 20, 2018

@tetreault what kind of mistake it was?.. I'm having similar issue

@tetreault

This comment has been minimized.

tetreault commented Jun 20, 2018

I updated my version of iOS @alienpavlov and when i did, for some reason, the javascript was disabled in my safari settings. No clue how that happened.

@tetreault

This comment has been minimized.

tetreault commented Jun 20, 2018

@alienpavlov here's my full JS demo if you need it:

"use strict";

document.addEventListener("DOMContentLoaded", () => {
  const startButton = document.getElementById("start-video");
  const frontCamVideo = document.getElementById("front-camera");

  let constraints = (window.constraints = {
    audio: false,
    video: {
      facingMode: "user"
    }
  });

  function handleSuccess(stream) {
    var videoTracks = stream.getVideoTracks();
    console.log("Got stream with constraints:", constraints);
    console.log("Using video device: " + videoTracks[0].label);
    stream.oninactive = function() {
      console.log("Stream inactive");
    };
    window.stream = stream; // make variable available to browser console
    frontCamVideo.srcObject = stream;
  }

  function handleError(error) {
    if (error.name === "ConstraintNotSatisfiedError") {
      errorMsg(
        "The resolution " +
          constraints.video.width.exact +
          "x" +
          constraints.video.width.exact +
          " px is not supported by your device."
      );
    } else if (error.name === "PermissionDeniedError") {
      errorMsg(
        "Permissions have not been granted to use your camera and " +
          "microphone, you need to allow the page access to your devices in " +
          "order for the demo to work."
      );
    }
    errorMsg("getUserMedia error: " + error.name, error);
  }

  function errorMsg(msg, error) {
    console.error(msg);
    if (typeof error !== "undefined") {
      console.error(error);
    }
  }

  startButton.addEventListener("click", e => {
    navigator.mediaDevices
      .getUserMedia(constraints)
      .then(handleSuccess)
      .catch(handleError);
  });
});

ssorallen added a commit to ssorallen/camtime that referenced this issue Jul 9, 2018

@PrItHvI7

This comment has been minimized.

PrItHvI7 commented Aug 27, 2018

Does Progressive Web Apps on iOS support “getUserMedia/Stream API” ? i am developing an application using Quagga barcode scanner, where it works fine on ios safari but while accessing through webApp it crashes with error below. [Error] ERROR — Error: Uncaught (in promise): TypeError: undefined is not an object (evaluating ‘k.inputStream.type’)

@tetreault

This comment has been minimized.

tetreault commented Aug 27, 2018

no @PrltHvl7, i hate to break it to you but apple either intentionally or unintentionally did not integrate webrtc into anything that uses UIWebview or WKWebview. It really sucks, i had to split up my web app so mobile safari and android use webcam and everything else on iphone (ios chrome/ff/social in-app browsers/stand-alone PWA) utilized a photo upload since those webviews don’t have webrtc support

@webrtc webrtc locked as resolved and limited conversation to collaborators Aug 31, 2018

@fippo

This comment has been minimized.

Contributor

fippo commented Aug 31, 2018

Locking this conversation which has been very useful and became the de-facto reference for this :-)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.