Don't require user gesture when capturing user media

[beaufortfrancois]

Video meetings web apps would benefit from automatic Picture-in-Picture behavior when user switches back and forth between web app and other applications/tabs. This is currently not possible with the user gesture requirement in https://wicg.github.io/picture-in-picture/#request-pip (step 6).

I'm proposing to not enforce the user gesture requirement if document is capturing user media with getUserMedia() when requesting Picture-in-Picture.

What do you think @mounirlamouri?

const video = document.createElement('video');
video.srcObject = await navigator.mediaDevices.getUserMedia({ video: true });

window.onblur = _ => { video.requestPictureInPicture(); }
window.onfocus = _ => { document.exitPictureInPicture(); }

---

Preview | Diff

[mounirlamouri]

My concerns with this is that it is very specific to some use cases and wouldn't allow a whole class of use cases to work. Also, I do not know about iOS but on Android, I do not think this could be implemented because the Android API expect Picture-in-Picture to happen synchronously when the activity gets hidden and the blur event would happen to late.

[beaufortfrancois]

My concerns with this is that it is very specific to some use cases and wouldn't allow a whole class of use cases to work.

It is specific you're right. However I'd like user to be able to jump on a video call by simply going to a website that would provide the best of PiP: autoPiP.

Ephemeral and seamless are key there and I think getUserMedia() provides a strong signal to allow this kind of interaction. And it is backed up by a user permission prompt already.

Do you have something in mind that would allow this for a whole class of use cases?

Also, I do not know about iOS but on Android, I do not think this could be implemented because the Android API expect Picture-in-Picture to happen synchronously when the activity gets hidden and the blur event would happen to late.

Sorry. I'm not sure I understand this argument. What prevents us to call Android code getActivity().enterPictureInPictureMode(); when Javascript code visibilitychange event listener fires for instance?

[beaufortfrancois]

@jernoble I'd love to hear your thoughts on this Auto Picture-in-Picture behavior we're trying to accomplish.

[jernoble]

I’m inclined to say that pip should always require a user gesture, and that loosening this requirement will lead to a slippery slope of sites with similar use cases demanding the restriction be lifted for them as well.

That said, iOS Safari does have an “auto-pip” behavior when an app is dismissed and video is being presented in full screen. This proposed behavior is pretty similar in effect.

Hypothetically speaking, if we did agree to enable this use case, I can see it working one of three ways:

1. Once gUM() is started, the page can call requestPiP() freely without restriction.
2. Once gUM() is started, the “visibilitychange” event will be considered a user gesture for the purposes of pip.
3. A site can opt into auto-pip on visibility change declaratively.

Option 1) seems ripe for abuse and even purely accidental bad behavior.
Option 2) may, as Mounir says, be too late to look correct.
Option 3) gives the UA enough information to correctly implement an “auto-pip on visibility change” feature without opening up a security or annoyance can of worms.

[juberti]

Note that auto-pip is probably needed on blur, not just on hidden, since IIRC alt-tab doesn't cause a hidden event.

[jernoble]

On the contrary, I’d argue auto-PiP is not needed on blur, for precisely the reason that the document isn’t hidden.

[juberti]

Well, neither are perfect; AFAICT, hidden does not fire when the browser is completely obscured by another window (and, in my testing, doesn't seem to fire on minimize either, but maybe that is a bug).

Blur at least fails safely in edge cases (showing the PIP and the document), whereas hidden fails more definitively, as nothing is displayed.

[jernoble]

Those both seem like bugs; Safari at least should fire a visibilityState change event when the window is fully obscured. Other UA may have made a policy decision not to fire change events when obscured, or the platforms on which they run do not elevate that information to apps. IIRC, the spec allows but does not mandate this behavior.

But to look at another angle, do we really want to allow pages to pop up a PiP window every time the user clicks away from the browser window? That seems like particularly annoying and potentially abusive behavior. That said, maybe some UA does want to do so; in that case, option 3 (declarative opt-in) would allow that kind of experimentation without mandating any specific behavior.

[juberti]

If visibilityState worked like you describe, I think that would be the best solution, since it really avoids all the problems I am concerned about without introducing the problems of blur.

[juberti]

IOW, for this spec, sounds like we should proceed in the visibilityState direction.

[juberti]

What are the next steps here? @beaufortfrancois, do you want to take a shot at a new PR regarding declarative behavior? @jernoble, did you have something specific in mind?

[beaufortfrancois]

@juberti I'm working on a new PR for a declarative behaviour. It will hopefully be ready for review really soon.

[beaufortfrancois]

@juberti @jernoble @mounirlamouri What do you think of this "Auto Picture-in-Picture" proposal at #111?
I'll close this PR so that we continue conversation in the other thread.