Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Picture-in-Picture #72

Closed
beaufortfrancois opened this issue Mar 14, 2018 · 42 comments
Closed

Picture-in-Picture #72

beaufortfrancois opened this issue Mar 14, 2018 · 42 comments
Labels

Comments

@beaufortfrancois
Copy link

@beaufortfrancois beaufortfrancois commented Mar 14, 2018

Request for Mozilla Position on an Emerging Web Specification

Other information

This specification intends to provide APIs to allow websites to create a floating video window always on top of other windows so that users may continue consuming media while they interact with other content sites, or applications on their device.

cc @mounirlamouri

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented Mar 14, 2018

For info, Picture-in-Picture got a LGTM at w3ctag/design-reviews#226 (comment)

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented May 30, 2018

We've announced recently our intent to experiment with Picture-in-Picture in Chrome. See https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/xQiDYZEnwaA/MNGbYJnaBwAJ

@jetvillegas
Copy link

@jetvillegas jetvillegas commented May 31, 2018

I spoke with a number of Mozilla folks who work in this area, including the folks who implemented the min-vid Firefox browser extension.

We're not opposed to the proposal, and agree that the use cases are valid, but we don't have bandwidth to implement or send detailed spec comments at this time. Please keep us posted on your implementation experience, and we'll take a closer look as engineers free up from other projects.

@dbaron dbaron added the w3c-cg label Aug 9, 2018
@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented Nov 5, 2018

FYI Picture-in-Picture has shipped in Chrome 70 for Linux, Mac, and Windows.
https://developers.google.com/web/updates/2018/10/watch-video-using-picture-in-picture

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented May 16, 2019

For info, we're currently thinking about the next iteration of this API for arbitrary content.
Is that something Mozilla would be interested as well?

@martinthomson
Copy link
Member

@martinthomson martinthomson commented May 16, 2019

@mikeconley had some great thoughts on this and might be able to offer a view.

@mikeconley
Copy link

@mikeconley mikeconley commented May 23, 2019

Sorry for the delay in responding. I'm actually going to redirect a request for comment to @marcoscaceres.

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented Jun 3, 2019

@marcoscaceres (gentle ping)

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Jun 3, 2019

reading it now... sorry for the delay y'all.

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented Jun 3, 2019

@marcoscaceres Thanks for the spec feedback! We'd be interested in your thoughts as well in the V2 proposal for arbitrary content: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/uK0hyACy_fg/JVXGUVylAAAJ as noted in #72 (comment)

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Jun 3, 2019

Tl;dr: I think we should "defer". However, if we want to ever implement a PiP capability in Firefox, we should do some serious UX prototyping and see if we can do this without needing an API. If it proves too challenging/impractical, we should definitely prototype the API. We could even start by just enabling the OS-level PiP in MacOS, to match Chrome and Safari (no API needed!) just to see if people would use it.

The long version: Having read the V1 spec... the spec is well thought out, but has a couple of little potential race conditions and underspecified things - but nothing that can't be easily fixed. The API is straight forward, and integrates well with various new security features afforded by the platform, like feature policy.

There is, nevertheless, a larger question if we need this API at all: Firstly, the use case for this capability (PiP) is undeniably useful - I personally used this capability a lot, both on my iPad and on my Mac. I've live-tweeted live events using PiP, and I've used PiP while watching educational videos, where I needed to see the video while I was coding a long to something... so I'm sure I'm not alone in seeing PiP's value.

Now, for the API, however, I've been able to use PiP in Safari without the need of API at all which really begs the question at to why we need this API at all.

On YouTube, using PiP in Safari and in Chrome, it's somewhat annoying for end-users: you need to right-click twice to get the native PiP option to appear. Having the API would allow PiP to be integrated into the custom right click menu (the spec also shows examples of this).

On an iPad, Safari overcomes the issues around UI by only making the PiP button available while in full screen (which only shows native video controls)... this is somewhat of a limitation.

So, from this - we should discuss with, as a I said above, I think we (Firefox) should discuss seriously with our UX folks what we can do here... and if we do go down the API route, we should go back and review if we need to balance out developer control over this feature with user's control over this capability (e.g., why should the site never allow a user to not do PiP for any video?).

@andreasbovens
Copy link

@andreasbovens andreasbovens commented Jan 23, 2020

if we want to ever implement a PiP capability in Firefox, we should do some serious UX prototyping and see if we can do this without needing an API.

Update: we've now shipped user-invoked PiP on Firefox for Windows, macOS & Linux.

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Jan 23, 2020

Ok, I guess we let this cook for a year or so, and then re-evaluate if we need the API.

@enjikaka
Copy link

@enjikaka enjikaka commented Feb 7, 2020

How about we stop cooking that right away? 🙈

I've just implemented PiP on https://listen.tidal.com. Being released next week.

Here the custom implementation from Firefox over the API proposal from Google that Chrome and Safari (Tech Preview) implements causes issues.

  • We cannot hide the icon
  • We cannot let users enter PiP through our own icon
  • The icon disappears randomly when switching between our different views (footer player <-> now playing <-> full screen). (The

Could you please use the same API instead? :)

Screenshots for context: https://imgur.com/a/ahDhjuR

@joaoBeno
Copy link

@joaoBeno joaoBeno commented Feb 10, 2020

How about we stop cooking that right away? 🙈

I've just implemented PiP on https://listen.tidal.com. Being released next week.

Here the custom implementation from Firefox over the API proposal from Google that Chrome and Safari (Tech Preview) implements causes issues.

  • We cannot hide the icon
  • We cannot let users enter PiP through our own icon
  • The icon disappears randomly when switching between our different views (footer player <-> now playing <-> full screen). (The element is moved to its new position here)

Could you please use the same API instead? :)

Screenshots for context: https://imgur.com/a/ahDhjuR

Lol... So Firefox, should just rubber stamp what Google Blink does and let them rule? The API is not approved, and Firefox is testing an opposing implementation that is better for the end user... My computer, my browser, my network, it surely should be my rules...

That's why I picked Firefox as my browser since Opera became Chrome...

@enjikaka
Copy link

@enjikaka enjikaka commented Feb 10, 2020

@ric2b
Copy link

@ric2b ric2b commented Feb 10, 2020

You can simply hide the icon when the browser doesn't support the API. Firefox users will know how to enable PIP. That fixes the dead button issue.

And honestly I prefer that my browser has a standard way to enable PIP that actually works on all websites. Imagine if there was a "back button" API so that websites could style it/etc, it would be awful. I don't think this is very different from that.

@enjikaka
Copy link

@enjikaka enjikaka commented Feb 10, 2020

@mozilla mozilla locked as too heated and limited conversation to collaborators Feb 10, 2020
@adamroach
Copy link
Contributor

@adamroach adamroach commented Feb 10, 2020

Thanks for the input. To be clear, this issue has to do with Mozilla's support of the API from a standards perspective. There is a separate project that tracks the implementation status of platform features.

I'd also like to remind people to look over the Community Participation Guidelines, and to keep their feedback polite and professional.

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Feb 10, 2020

I've locked this thread until things cool down a bit.

@enjikaka, we really appreciate your input, and understand your frustration. We are trying some new things and they may not work out (with the UI we've been experimenting with). This is not an us vs them thing - but something we'd still to explore. We are constantly re-evaluating our assumptions, and having you show us how our UI doesn't work on your service is a helpful data point. However, statements like "buhu they didn't come up with it first" don't help make your case, and act as a disincentive.

@mozilla mozilla unlocked this conversation Feb 25, 2020
@JeremiePat
Copy link

@JeremiePat JeremiePat commented Mar 3, 2020

Hi!

Here's a quick feedback about that new feature in Firefox.

First of all, from a user perspective, thanks for bringing that awesome feature. It was something missing and it's great to have it available in Firefox at last.

That said, from an author perspective, I have to say that I'm quite confused.

First of all, I do not understand the heuristic to display the PiP UI. On a simple local test page with a single <video> element, the UI never show up where it appears on any video web site when overing a <video> element. So, first discovery: I cannot predict when that UI will show up to my users! 😥

Second, the current blue "native" UI widget is an excruciating scare on many UX.

The trick is that it's possible to somewhat work around that issue by playing the video within a canvas element 🙄. It's really unnecessarily time consuming but that's not the biggest problem. The true problem is because there is no API, authors can hide that questionable UI if they really want to, but in exchange, they cannot expose that feature back to their users in their own terms.

I could understand not being able to customize the PiP window content (even if it's not 100% satisfactory), but not being able to customize the UI to access it, that is a hard no go for me. What if the whole UI for the <video> element was not customizable? There is no good reason to stop authors for customizing a part of that UI, which is currently the case with PiP. I don't really understand the reasoning to not expose at least a minimal API to let authors create their own UI on top of that feature (at least as minimal as the requestFullscreen API).

So basically, the feature is here (and the user in me say thanks again) but it cannot be managed to suits authors expectation, which is just another reason to, sadly, dismiss Firefox.

@enjikaka
Copy link

@enjikaka enjikaka commented Mar 4, 2020

It is also weird that a custom control from a browser vendor does not end up in the <video>-native controls, and also does not listen to the controls attribute. This whole implementation is badly planned from start to finish.

Even when the attribute is absent, however, user agents may provide controls to affect playback of the media resource (e.g. play, pause, seeking, and volume controls), but such features should not interfere with the page's normal rendering. For example, such features could be exposed in the media element's context menu.

https://www.w3.org/TR/2011/WD-html5-20110113/video.html#user-interface

I'd say that Firefox custom PiP button very much interferes.

Mozilla is clearly not considerate of developers. I'll stop using, recommending and caring about making the stuff I develop work in FF from now on.

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Mar 4, 2020

@enjikaka we understand you are frustrated, but insulting the work of Mozilla developers won't help advance your case. Kindly ask that you follow our participation guidelines and be respectful. No one is saying we won't implement the API - we are just not doing it right now. Even if we agreed to implement the API, it could take us a while to get to it.

@enjikaka
Copy link

@enjikaka enjikaka commented Mar 4, 2020

@marcoscaceres If you are not going to implement it right now, at least swiftly remove this bad alternative you've made up. You're welcome to add a custom PiP-button to the native video controls or the context menu, as per the spec.

@marcoscaceres
Copy link
Contributor

@marcoscaceres marcoscaceres commented Mar 4, 2020

@enjikaka, respectfully, it's our browser: We don't walk into your home and start telling you how to decorate or what you should eat. We really appreciate that you've brought issues to our attention that may negatively be impacting users and developers, but please refrain from telling us what to do.

@beaufortfrancois
Copy link
Author

@beaufortfrancois beaufortfrancois commented Mar 4, 2020

Do you think it would make sense to add partial support for the Picture-in-Picture Web API by simply supporting the disablepictureinpicture HTMLVideoElement attribute at https://w3c.github.io/picture-in-picture/#disable-pip

This way, the browser PiP button would be hidden for a video element if web developers set its disablepictureinpicture attribute to true. This would be as easy as below to provide a nice user/developer experience.

if (!document.pictureInPictureEnabled) {
  // Picture-in-Picture Web API doesn't seem to be supported.
  // Let's add a hint to the browser that a PiP browser button is not suitable.
  document.querySelector('#myVideo').setAttribute('disablepictureinpicture', true);
} else {
  // It's up to web developers to allow or disallow PiP at this point.
}

@JeremiePat
Copy link

@JeremiePat JeremiePat commented Mar 4, 2020

Hey folks, I think everybody should cool down a bit here.

@marcoscaceres I totally agree with you that Mozilla takes whatever decision regarding their product. However, knowing Mozilla process reasonably well, I'm a bit surprise that, in its current state of implementation, that feature ride the train out of nightly so fast. Nightly could have been used to get the feedback regarding that feature in a way that would have been more peaceful. It's true that the prose of @enjikaka is, let's say, "spicy" in a way that is not really helping but I also understand his frustration very well: As a web author, the decision made by Mozilla to ship a feature that could be easily qualified as "half finished" and that introduce hard UX discrepancies directly within my carefully craft web content is infuriating. I urgently need to workaround a tricky unwanted UX change which has a direct impact in my own product roadmap, forcing me to postpone other features and therefor impact my business. In such a context I think it's reasonable to "let it go" when people start to be a bit harsh on comment.

So yes, as web author, we do not have to tell Mozilla what to do, but do not expect any thanks if you screw up our own product roadmaps and don't be surprise if we, web authors, just decide to give up on a browser that jeopardize our own business. To conclude, even if it's very important to have civilized discussion about web features, I'm not sure that Mozilla is currently in a position to lecture web authors when Mozilla is taking decision that impact their business which, very understandably, will piss them off.

@JeremiePat
Copy link

@JeremiePat JeremiePat commented Mar 4, 2020

@beaufortfrancois

Do you think it would make sense to add partial support for the Picture-in-Picture Web API by simply supporting the disablepictureinpicture HTMLVideoElement attribute at https://w3c.github.io/picture-in-picture/#disable-pip

I haven't dig deep within the spec, but could you remind what is the reasoning to introduce a disablepictureinpicture attribute instead of having the feature just being toggle like the rest of the <video> controls with the controls attribute`?

From my web author perspective having PiP reacting to the controls attribute would be good enough as I'm usually disabling all controls to build them up back (and the whole flame in that topic is clearly about the UI issue rather that the whole feature itself). Regarding the current Mozilla implementation I would be satisfied if they would be okay to have the PiP UI honoring the controls attribute.

@enjikaka

This comment was marked as off-topic.

@mikeconley
Copy link

@mikeconley mikeconley commented Mar 4, 2020

If it helps at all, we've added the ability for us to reposition the toggle for sites on a particular domain. If the toggle is interfering with a piece of UI (for example, if it's intersecting a semi-transparent button and intercepting user clicks unexpectedly), we can position the toggle at different places along the right-edge of the video.

In the event that you'd like us to evaluate repositioning the toggle on your site, please file a bug on Bugzilla blocking this metabug. Alternatively, you can email me at mconley at mozilla dot com with a link to the site for us to test, and I'll file the bug for you.

@enjikaka
Copy link

@enjikaka enjikaka commented Mar 4, 2020

@mikeconley Any position for the PiP button within <video> breaks the UI on https://listen.tidal.com in the footer player as there is a custom hover control there to open an alternative player view:

@mikeconley
Copy link

@mikeconley mikeconley commented Mar 4, 2020

This looks rather like a bug - we attempt to hide the toggle on videos that are smaller than certain dimensions. This video should qualify for that exemption, but it's not. I've filed this bug for it.

We should move discussion of that issue to that bug, and try to keep this GitHub issue focused on the proposed PiP WebAPI, and Mozilla's position on it.

@jimmywarting
Copy link

@jimmywarting jimmywarting commented Mar 23, 2020

Just discovered pip was available in FF (chrome user here)
recently read How we built Picture-in-Picture in Firefox Desktop - (grate article btw)

  • From one perspective I like that you have made pip available for all the videos out there but as a developer I would like to style/control it myself.
  • I think a better approach would have been to develop something like the chrome pip extension that detects videos and lives outside of the DOM instead. and interacting with the OS instead like the touch bar for example.

It's sad to see 3 different implementation that all behaves differently. I understand it's good to be competitive and design new things to come up with something better

but i hope later that all can agree on one unify spec:ed version later - (whichever it may be)


I would not mind either if you kept it like the way it's right now. But if i detect that it looks bad on my website I would like to disable it and reimplement it myself - hence why i think a API would be useful

One thing i like about chrome + safari's implementation is that i can first detect when it enter/leaves the pip mode so i can style the wrapper element and have it look the same. for instance i set the video poster as a background (still have the video element on top with a opacity: 0 so they can still control it with the context menu also.) and i can let the custom control panel be visible at all time

(Partial support for the Picture-in-Picture Web API don't sound that bad either)

Google Dev blog had a good usecase also: Show canvas element in Picture-in-Picture window

const video = document.createElement('video');
video.muted = true;
video.srcObject = canvas.captureStream();
video.play();

// Later on, video.requestPictureInPicture();

what if it's a offscreen video element and the only visible is the canvas element? then there is no way to enter pip mode. unless you switch out the canvas element for the video element instead.

@mikeconley
Copy link

@mikeconley mikeconley commented Apr 17, 2020

This looks rather like a bug - we attempt to hide the toggle on videos that are smaller than certain dimensions. This video should qualify for that exemption, but it's not. I've filed this bug for it.

Following up here, this bug has been fixed in the latest Firefox Nightly, and should ride out in Firefox 77 (should release June 2nd).

@theSdev
Copy link

@theSdev theSdev commented May 6, 2020

I see the problem has to do more with the UI of the button than the lack of api, and saw a couple of suggestions to fix this in this issue, and am surprised that nobody has suggested a psuedo element for styling the button (likes the ones form controls have).
Can Mozilla evaluate this approach?

@graywolf
Copy link

@graywolf graywolf commented Jun 20, 2020

@theSdev wouldn't developers just style it as display: none or something and call it a day?

From the user perspective, I love how this is available everywhere and I don't have to hope that developers don't disable it for whatever idiotic reason.

The blue button can be hidden, there is option for that in settings so users that do mind it being there can just turn it off no?

@jculvs
Copy link

@jculvs jculvs commented Jul 11, 2020

Any updates to this? I am seeing that the PiP does not show up for videos smaller than 45 seconds or so. Is there possibly just a way to change this threshold?

@mikeconley
Copy link

@mikeconley mikeconley commented Jul 11, 2020

@jculvs If you set media.videocontrols.picture-in-picture.video-toggle.always-show to true in about:config, the toggle will skip most heuristics and always show.

@bkniffler
Copy link

@bkniffler bkniffler commented Jul 11, 2020

This is not directly related to the UI issues that may be caused by PiP as discussed before, but more general PiP.

Whats the reason for FireFox (and all other major browsers) to implement PiP in a custom way? All OS now have native PiP capabilities that are streamlined perfectly into the OS experience.

Advantages of native PiP as seen in MacOS Safari:

  • Low CPU/GPU costs (compared to FireFox)
  • Works across workspaces and even over fullscreen applications
  • No need to maintain custom code
  • Consistent experience with other PiP on the same OS

Advantages of custom PiP

  • Only one video at a time
  • Limited possibilities for positioning (corners)
  • Consistent experience across different OS

I greatly prefer the native experience, but this might of course be a personal preferences. Whats the opinion of others around here?

@jculvs
Copy link

@jculvs jculvs commented Jul 13, 2020

@mikeconley I understand there is a way to adjust this as a user but I am serving video that I wish to disable PiP on. Most videos I serve are shorter than 45 seconds but a few go over that default threshold and if the users goes into PiP it breaks other features on my page. Most browsers allow me to disable this easily, is there any option like this on firefox as of now?

@mikeconley
Copy link

@mikeconley mikeconley commented Jul 13, 2020

@jculvs No, there is not a way for sites to control whether or not the toggle appears.

@jonnyfux
Copy link

@jonnyfux jonnyfux commented Jun 11, 2021

Any updates on implementing the API? I would really love to use requestPictureInPicture() method in Firefox.

I think there are a lot of use cases where it makes sense to support this feature, to just give my example: We offer a website with a live stream and other components with more information. If the API would be supported, the user could "take" the live stream with them while browsing through the site. The PiP mode would be the perfect solution for this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.