[mediaqueries] length units in video-* media features. #5044
Comments
|
Would it be possible for you to expand upon "anchor the css px unit to physical device pixels rather than to the angular definition." Does this mean that the pixel definition is normally related to angle subtended (and hence, viewing distance)? |
|
The term anchored also explains which definition to use n which circumstance. However, it only defines two options and this specification uses a third, which should be added to V&U 4:
add
|
Why would this be okay for |
|
The CSS Working Group just discussed The full IRC log of that discussion<dael> Topic: [mediaqueries] length units in video-* media features.<dael> github: https://github.com//issues/5044 <dael> florian: As we know length in CSS are interesting. All units fixed ratio to each other and either to phsyical measurements or pixel. For length in video what are we expecting? <dael> florian: I think a third one. video width and video-height if this is 4k I'm expecting an answer in terms of pixels. <rego> dholbert: still you get overflow there, in both directions (not very useful to me); and also you can have the very same case in flexbox https://jsfiddle.net/h8u53ows/ (that will work different only for row gaps) <dael> florian: I don't think we discussed use case and usage in much detail. Do we expect video plane into angular css pixel? Maybe, but if that's the case do we need this? If it's the phsyical pixel we need to write that <dael> nigel: For using video pixels I think something exists in TTML specs where you can define root container region and say how big it is in pixels. If you're not doing that you get pixel resolution of video. So you can spec location and size which defines where you want stuff <dael> nigel: Similarly define font size in pixels. Very typicaly to use video pixels when authoring in that sense <dael> nigel: Bit of a push away b/c makes mroe sense to use relative dimensions to the rendering area as a %. For font size calc became complicated using % fonts based on parent element. <dael> nigel: We introduce rw and rh which is relative to rendering area, not overall page <dael> florian: Another confusing thing is we're defining width and heigh of video plane but not saying what's in there. Assumption is that's only hardware generated so you wouldn't size fonts, but maybe 'm wrong? <dael> chris n: I think you're right. In full screen we're interested in knowing device resolution so we can select appropriate media representation for streaming <dholbert> rego, (right, there's overflow, but it's consistent/symmetric for grid there, and it doesn't have any percentages depending on content-measurement, which makes it a case that feels less terrible. *shrug*) <dael> chris n: returning in device pixels is ideal <dael> florian: But for images we're not doing that but asking API to describe the sources and let the UA pick the appropriate one. Are we not doing that here? I believe you can desc multiple sources, no? <dael> chris n: If you're using video element you could. If your'e using media source extensions that becomes web app's responsibility not the UA's <dael> Rossen_: We're at top of hour <dael> florian: I think I have a direction, but we should explore on github <dael> Rossen_: I would encourage chris and nigel to interact on that issue. You've got the people to solve this |
|
Some questions:
|
To be of any use, it would need to be the size of the window I think, but then again, are there devices/OSes which use the notion of a video plane and have non full screen windows?
That's the key question, and based on the discussion on the call, what I think I'm hearing is that the video-width would be calculated at 1x, so on a 2x Retina screen, video-width would be twice as big as width. But that's not a clean fit with how the rest of css works… |
|
If video-width is to be measured in device pixels, it should not be using |
|
Device pixels are needed for the video plane so that the web app can select the correct video resolution. It's important to avoid rescaling the video during playback, where possible.
For TV devices, I'd expect this to still return the native resolution of the full screen video plane. We can use existing media queries for the browser window size.
This sounds good to me. cc @mwatson2, @vi-dot-cpp, @chcunningham for additional input from Media WG members. |
|
+1 to @chrisn's comments. We need to decide which version of the video to stream. If the device has enough physical pixels on the screen to make delivery of 4K worthwhile, we'll stream that, even if the window is not - now - fullscreen but might be later. We want to build up a video buffer, so we request media in anticipation of future needs. There is a slight difference here for video compared to how you might handle images where you might download the resolution you need now and fetch a new one if the window is resized. If the device does not have enough physical pixels to make 4K worthwhile, we'll deliver HD as this would be an overall better UX (less chance of rebuffering, greater possibility to avoid scaling artifacts). The desire above applies whether or not the device has a separate video plane architecture, but I think there are sufficient properties already to determine the physical pixel resolution in the single-plane case. |
|
The CSS Working Group just discussed The full IRC log of that discussion<dael> Topic: [mediaqueries] length units in video-* media features.<dael> github: https://github.com//issues/5044 <dael> florian: I can remind where we're at. Not sure how to solve. <dael> florian: We have a set of MQ which are specific to video. I'm going to focus on width but have height too <dael> florian: Just like normal webpages render to viewport, some players render video to a separate plane. We have video-width and like that <dael> florian: Proposed with same syntax as normal width and height. On devices where planes are different they accept same syntax. THat implies they take any kind of length and what do those units mean? Goal is the for a 4k device it would return 4k. <dael> florian: Should it be pixel with a different def and if so what do the others mean. SHould it be a unitless, separate unit. Underlying that is how are they expected to be used <dael> TabAtkins: For the use case people want to know phsyical density of screen. If it's a 4k screen they want 4k video no matter if it's full screen <dael> TabAtkins: Given that I agree with fantasai we should do an int <astearns> q? <hober> q+ <astearns> ack hober <hober> q? <dael> hober: I think I would like there to be a princple for answering questions like this. Here on a device with only one plane in a UA that supports video prefix they should be same as if alias for eq. <dael> florian: That's where we started but introduces problems <dael> TabAtkins: Explain why useful principle? <dael> hober: Sure. I think...I worry this 2 plane model is a contingent fact of present limited hardware. <AmeliaBR> q+ <florian> q+ <dael> hober: I would like in the future if this hardware goes away I would like the maintence burden for browser to be minimized. Great if simple alias. Working backwards is how I get here. <astearns> ack AmeliaBR <dael> hober: Future is longer than past and I doubt devices with these characteristics are a thing in 100 year, but I hope CSS is a thing in 100 years and I hope we don't burden future impl <emilio> hober++ <dael> AmeliaBR: Following up on hober comment, I do agree we shouldn't have similar but different things. Px with a different interpretation is dangerous. <dael> AmeliaBR: I don't know if that meanst hey have to be alias but we need to think question and purpose of this MQ. If we do the int value where it gives you raw dots on the screen we need to ask how is that interp by all UAs, even ones not doing special video rendering <dael> AmeliaBR: Does defining facotr become this isn't about sep plane but this is a way to get raw pixel resolution to render graphics and will that be expected in all UAs? <dael> AmeliaBR: If it isn't just an alias for existing width and height we have to talk about what does it means and what's the purpose <astearns> ack florian <dael> florian: A few things. Minimizing maintainence burden is good. If we define this as non or similar I think it's not heigh. <dael> florian: Mapping to an alias is odd. You want ot deliver 4k on a 4k screen you don't want to deliver 4k if there's one plane. <AmeliaBR> `<video><source media>` <emilio> What is the expected usage of this media query? <dael> florian: Starting to think is it useful as a MQ. It's not to select stylesheets, it's for API things that want to know screen and download assets. Wouldn't this be better as an API? <dael> AmeliaBR: MQ are used to select video sources in HTML attributes <dael> florian: For that purpose I think the best practice is not to select on device resolution but desc resolution of videos and let UA choose. This is for the cases you can't do that b/c managing it yourself in JS <dael> astearns: florian you're suggesting? <dael> florian: srcset is better for that. I'm thinking this isn't a MQ and this number is exposed in API <astearns> ack fantasai <dael> fantasai: Also a question of this is the device...viewport size not device. <dael> florian: It's device <plinss> q+ <dael> fantasai: Desc number of pixels or length of viewport. Question if use case if for viewport size or device size. Need to be clear if we need one or both for the use case <dael> fantasai: We could make something only available in JS but still seems fund. a MQ. Doesn't make sense to create and API that's ntoa. MQ even if we think it's only useful in JS <faceless2_> q+ <astearns> ack plinss <dael> florian: MQ don't give numbers, they answer a question. If people want a resolution they have to search the MQ. Seems not great <dael> plinss: Orthogonal, I want to object to 4k or 4000 because it's a marketing term not an exact specification. It is a hot mess so let's not use it in a technical context. <astearns> ack faceless2_ <dael> faceless2_: florian people are biary searching MQ to get approx resolution already. Ultimately if you're prop a value to desc phsyical px I can think of multiple issues over the last 6 months that wanted that. IT's a useful problem to solve and not restricted to video <dael> florian: Then different q. Number of phsyical pixels across the screen on the video plane is different than one across viewport. <dael> faceless2_: Device with a single screen is it different? <dael> florian: Yeah, if browser not full screen it's different <dael> nigel: I got impression idea is combo of queries for that. if you have info about whole device video plane you can do other queries to realize actual viewport is a subsection of that <dael> astearns: We have video width return px on video plane nad other MQ gives you px width of browser window and we can't convert between I'm not sure how useful <dael> florian: resolution gives density and that's in MQs <florian> q? <florian> q+ <dael> AmeliaBR: One thing to mention, I can't pull exact resolution but org discussion is MQ were part of solution but also going to be CSSOM API that exposed exact numbers. Screen API and screen.video <dael> AmeliaBR: That people want exact numbers doesn't mean MQ are inappropriate. Might be 2 parallel solutions. Can't remember what we resolved on <emilio> faceless2_: hmmm, why are people bin-searching media-queries instead of `window.devicePixelRatio` / `window.screen` / `window.{inner,outer}Width`? <faceless2_> A fair question, I didn't write the library that did it :-) Will dig it up and post here <dael> florian: Another comment, we used to have device-width and -height MQ in CSS px but we deprecated them for privacy reasons. If we intro this we're re-opening hte problem. It's a trade-off so I don't have a decision, but want to remind people <dael> smfr: Strong support florian. For srcset the UA picks the source and we should push in that direction so we don't have to expose all device information b/c fignerprinting <dael> chrisN: Like we discussed src requires knowing the device characteristics to know what media to pick <dael> florian: API to tell what resolutions are available and you can than tell which to load? <emilio> q+ <dael> ChrisN: potentially <dael> florian: With an observer of some kind so if it moves from high to low density screen you get told <dael> ChrisN: Feels like need to take to media WG. Looking at history and we had 3 options, one of which was to add properties to screen object. <dael> AmeliaBR: I was reviewing that discussion and last comment was someone saying open a separate issue to discuss how API should look and I don't think that issue was ever filed <florian> q- <astearns> ack emilio <nigel> q+ <dael> emilio: I feel like florian and smfr. It doens't feel to me like using a MQ is right. For video there's a lot UA would want to do. Seems sensible that even if high res screen you want to take low bandwidth into account. Leaving source decision to UA seems better to me and not clear how use MQ <Rossen_> strong +1 <dael> florian: I suggest we leave this open, go back to media group to talk and with a suggestion to look at discussion for images and how srcset came to be because that was a similar item. People should look and have in back of mind. Doens't mean we have to do same but it's releated <astearns> ack nigel <dael> nigel: I want to observe there's an archetectural side. THere's a bunch of MQ about video and some make sense in API and others don't. Need to think about if we want consistent across all or if it makes sense to differ. Video size is an obvious thing to decide on resource. Another might be indicate if display can do a color gamut where you can choose a different version <dael> nigel: Might also want to select different colors for text in that case. <dael> nigel: It's complicated and would be neat if single approach to extend. <dael> florian: We do have color gamut and dynamic range for video. Width and height seem harder <fantasai> s/video/video. These seem fine to me./ <dael> nigel: iF users have to go to different places for info about same device that makes their job harder <dael> florian: But it's different. Picking different color of text you would do in CSS so a MQ is appropriate. If math in JS to figure out resource not obvious MQ is convenient. <dael> nigel: Yeah, just different things to balance <dael> ChrisN: I think that's how we ended up with this propoal. We have color gamut, let's put these in same place. Can revisit <dael> astearns: On face it seems reasonable peolpe have shown issues with this MQ, esp with privacy. Appropriate to go back to media QG and see what is really required for MQ. Example usage is great <dael> hober: Joint call might make sense and be faster than going back and forth. Media WG telecon is monthly <dael> astearns: Excellent idea <dael> astearns: Possible to invite CSS people to media WG call? Or should we host? <dael> hober: Imagine either <dael> nigel: I don't think we have Media group on the call <dael> ChrisN: It could work either way and may depend on timing. <nigel> s/Media group/Media group [chairs] <dael> florian: Would be good in same discussion to have someone, maybe TAG who can represent responsive image discussion <dael> hober: With my TAG hat on I guess I'll have to be on <dael> TabAtkins: Same here since I'm also responsible for srcset <dael> astearns: I'll reach out to Media WG chairs to get a time for a joint call <dael> florian: Next issue is related but different |
|
Hey Gang, I want to follow up from the CSS : Media WG call (minutes). The call helped me to understand some issues with specifying that video- queries use device pixels: 1) fixing resolution to 1 dppx is unusual 2) what do we do with 'em' 3) what do we do for fullscreen vs not I have a few ideas. I'm not an expert in CSS units... hopefully this isn't too naive. What if we spec that video-* uses css pixels just like non-video, and just leave implementers with the option to setup their video plane w/ resolution = 1dppx such that this is effectively device pixels where desired. IIUC, this isn't really novel. Implementers of existing media queries already enjoy this flexibility. 1dppx is valid and even common. The question of 'what do for em/fullscreen' is then answered: same thing we do already under these conditions elsewhere. I think this is also more correct for non-TV implementers. When you don't have a separate video plane, the simplest design is that video-* answers match that of non-video. I do see a potential wart (but I think its manageable). The earlier idea to spec that its actually device pixels makes media queries straightforward for callers. Folks like Netflix have a fixed number of resolutions available for a given title, so they only need a handful min-video-height queries). But if resolution is allowed to vary, it seems you have to binary search that value first to know what scale factor to apply. Is there some MQ syntax sugar that makes this easy? Even a true binary search seems pretty doable. Alternatively, we could take all of the above arguments about using CSS pixels and flexible resolution, and carry those to a proposal for Screen attributes. No binary search. On the call, I think a rep from Apple (was it @smfr ?) indicated this raised fingerprinting concerns. The binary search for this info is pretty trivial, so MQ seems not meaningfully more privacy protecting? |
|
Hi group. Just checking in on my last comment. Happy to meet to discuss further if that helps. |
|
@zcorpan @svgeesus @gregwhitworth @tabatkins @frivoal and I chatted virtually. They prefer adding video-width/height to the Screen interface as opposed to MQ attributes. The Screen path avoids users having to binary search for the value. Discrete attributes like dynamic-range and color-gamut don't have this pitfall, so these still make sense as a MQ attributes. There are still some details to sort out.
First thought: Add videoWidth and videoHeight to Screen. These return the full size of the screen, similar to existing width and height attributes, but using device pixels to describe the video plane. Seems ok for full screen video cases, but its not great for video that's inline among other elements. Maybe TVs only render video in full screen (@jpiesing let us know)? Is that something we can rely on going forward? Second thought: add deviceVideoPixelRatio to Screen. You can then determine the "video" pixels of anything, including screen (width, height), window (availWidth, availHeigh), as well as inline |
|
Do you need this info for all screens? https://webscreens.github.io/window-placement/#dictdef-screeninfo |
Mostly, but they also have picture-in-picture for thumbnail videos etc |
Probably only necessary for the "current" Screen (so current Screen semantics should work). Would be nice to have an event if the screen changed though. |
|
There's a related issue on the Window Placement API: w3c/window-management#20 |
@chcunningham Unfortunately it's more complex. TVs will;
Thjs is a little out of my area of expertise & this explanation may not be 100% correct but it's close ... |
|
Sorry for the long pause. @frivoal reminded me this remains open. Lets fix it! Given @jpiesing's comment above, I'd like to pursue the second option I suggested:
Only I guess I should revise that add the attribute to Window, not Screen (since that's where devicePixelRatio ended up). Authors would then write code like so... full screen case inline case And then fetch video assets according to whatever is closest to targetResolution. @smfr @zcorpan @emilio: would you mind if I send a PR to add this to cssom-view? |
|
Agenda+ to discuss the above proposal |
This direction looks good to me.
I agree. I don't think new events are needed if applications can use the |
|
The CSS Working Group just discussed
The full IRC log of that discussion<fantasai> Topic: MediaQueries length units in video-*<fantasai> github: https://github.com//issues/5044 <fantasai> florian: Issue has evolved since opened <fantasai> florian: reminder, we specified video-width/height/resolution <fantasai> florian: video prefix is about video playing that some devices like TVs have, where they superimpose a special canvas with different characteristics for playing video <fantasai> florian: About a year ago we were talking about this and thinking that actually, maybe MQ isn't what's wanted <fantasai> florian: The typical query of width is about whether bigger or smaller than some value <fantasai> florian: but not used for that <fantasai> florian: use cases were that ppl wanted to ask "what is the size of the screen, in physical pixels" <fantasai> florian: The only way to do via MQ is via binary search in MQ, which is not great <fantasai> florian: So conclusion of discussion is not to have an MQ, and instead have some extension to CSSOM where you can query for the pixel ratio of the video playing of the device <fantasai> florian: since can already know size of viewport, can get the size of the whole screen <fantasai> florian: or if interested in size of an element, can go from the size of that element <fantasai> florian: So the suggested resolution for MQ is to remove video-width/height/resolution MQ <Rossen_> q? <fantasai> florian: and another resolution would be for CSSOM to add window.deviceVideoPixelRatio <fantasai> Rossen_: Opinions on this? <fantasai> smfr: I'm a little concerned about multi-screen systems <fantasai> smfr: window is associated with a single screen <fantasai> smfr: if showing video, is it on the same screen/window? <fantasai> smfr: idk how to know that <fantasai> smfr: imagine you have two screens, right hand is dedicated to video presentation <fantasai> smfr: somehow magically when you present video, it shows up on the right <fantasai> smfr: but window object is associated with the left hand screen <fantasai> smfr: so would be weird if it reported characteristics of right hand screen <fantasai> ???: Is this possible today? <fantasai> smfr: UA thing, UA could decide to send fullscreen video to a particular screen <astearns> s/???/chrisneedham/ <fantasai> florian: I think as far as TVs are concerned, and things with this multiplane rendering, it doesn't behave the way you described <fantasai> florian: more general browsers/setups, could be other setups <fantasai> florian: the window object is probably not designed for that, but this is a more generic problem than video specifically <fantasai> florian: 2nd Screen effort is trying to deal with that <fantasai> florian: but for the TV use case, I don't believe that's how they work <astearns> s/chrisneedham/cpn/ <fantasai> Rossen_: to go one step further, in conjunction with some of the newer foldables and dual-screen efforts <fantasai> Rossen_: I know of at least one effort that is work happeneing on ??? that have to do with dividing up the window into multiple segments <fantasai> Rossen_: multiple screen divided by segments or hinge <fantasai> Rossen_: here we have one window spanning two surfaces <fantasai> Rossen_: video having controls on one different screen is very typical <fantasai> florian: Does this device have a multiplane rendering thing? <fantasai> florian: typically TVs do this, typical screens don't have two planes that can be queried separately <fantasai> Rossen_: we have one window object that will be present on both physical screens <fantasai> Rossen_: is expectation that devicePixelRatio returns ... <fantasai> florian: I think the question is legit, if you have this multiplane thing with two resolutions on one screen, we have this problem for the window object in general <fantasai> florian: this is a good problem to address if such devices exist, but there's nothing specific to video about it <fantasai> Rossen_: there are some laptops that have a pane over the keyboard, so you can present into those <fantasai> Rossen_: but might be represented by two different screen objects <florian> fantasai: don't smfr 's concern apply to the regular to the regular device pixel ratio as well? <florian> fantasai: if so, we need to address it there as well <fantasai> smfr: devicePixelRatio doesn't have this same issue <fantasai> smfr: if I drag window over, devicePixelRatio will change <Rossen_> fantasai: I understand the concern. We need to address the multi-resolutions for devicePR and other API. Adding a parallel API for these is what makes most sense. <fantasai> smfr: issue with video one is that video presentation is special, and UA chooses that it will be displayed in magic extra plane with different properties <fantasai> florian: We're not talking about picture in picture or displaying window in other places <fantasai> florian: but for TVs, they have two framebuffers layered over each other in the same spot <fantasai> florian: They have the normal layer transparent, punch video through to underlying layer <fantasai> florian: which has different resolution <fantasai> florian: in current devices, this would be on the same scree <Rossen_> ack fantasai <fantasai> florian: a browser could have two different videos on different screens, but that's a very different setup than here <fantasai> smfr: I understand, I just don't want us to design a problem for foldables <fantasai> florian: Let's just resolve to remove the MQ for now <fantasai> florian: and maybe we can open an issue for the deviceVideoPixelRatio <fantasai> florian: I don't quite see the problem, because whatever problem you're describing seems to apply equally to devicePixelRatio <fantasai> Rossen_: Anyone from the video group want both resolutions? <fantasai> fantasai: smfr, why does your concern apply to deviceVideoPixelRatio and not devicePixelRatio? <fantasai> smfr: Because it applies to fullscreen, which is more likely to be on a different screen <fantasai> florian: This isn't necessarily about fullscreen video. <Rossen_> ack fantasai <fantasai> florian: if you have youtube playing in the middle of your web page, you would apply this ratio within the video <fantasai> smfr: No, I don't think so, I think it only applies to fullscreen <fantasai> Rossen_: If presentation spans multiple screens, e.g. projector, very different characteristics than PC, wouldn't problem apply to that as well? <fantasai> florian: And doesn't necessarily apply to fullscreen only, that's up to the author <fantasai> florian: Yes, youtube or hulu or whatever video service can be played fullscreen sometimes <fantasai> florian: but might also want to query in normal rendering mode <fantasai> florian: and since hardware accelerated, would still go into special plane <fantasai> smfr: wouldn't be the case in Apple devices <fantasai> smfr: I think we should ask experts for feedback <fantasai> smfr: I think it would be very confusing for author <fantasai> Rossen_: Let's go with resolving to drop the MQ <fantasai> Rossen_: it's clear we need to work more on the rest of the API <fantasai> Rossen_: Any objections to dropping MQ? <fantasai> smfr: I support that <fantasai> RESOLVED: Drop video-width/height/resolution media queries <fantasai> Rossen_: send rest of conversation back to the issue <fantasai> florian: Should I retitle the issue, or file a new one? <fantasai> Rossen_: new issue might be helpful to have clear slate, but history etc. might be useful |
|
Sorry, I was on the call but had mic issues. I don't object to removing these media queries. I've filed #6891 to host the follow up discussion. |
The definition of the
video-widthmedia feature (same thing forvideo-height) includes this:This means (I believe) that we're doing things like measuring in CSS px, not in device pixels, as we always do everywhere.
However, I suspect that this isn't actually what implementations plan to do, and a 4K TV probably wants to report 3840 × 2160 (and a resolution of 1dppx), regardless of viewing distance.
If so, we should probably change the above phrasing, or at the very least add a note saying that implementations are likely to anchor the css px unit to physical device pixels rather than to the angular definition.
If we do intend the usual unit gymnastics, the above phrasing is fine, but a note should probably be added, as this may be unexpected / surprising.
The text was updated successfully, but these errors were encountered: