Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media controls #23208

Open
wants to merge 18 commits into
base: master
from

Conversation

Projects
None yet
9 participants
@ferjm
Copy link
Member

commented Apr 16, 2019

This is still highly WIP. It depends on #22743 and so it is based on top of it.

The basic controls functionality is there, but the layout is highly broken. There is a hack to at least make the controls render on top of the video, but it is not correctly positioned. The controls' div container ends up as sibbling of the media element in the flow tree while IIUC it should end up as a child.

  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • These changes fix #22721 and #22720

There is at least an extra dependency to improve the functionality and visual aspect: #22728.


This change is Reviewable

@highfive

This comment has been minimized.

Copy link

commented Apr 16, 2019

Heads up! This PR modifies the following files:

  • @asajeffrey: components/script/dom/htmltextareaelement.rs, components/script/dom/servoparser/mod.rs, components/script/dom/webidls/DocumentOrShadowRoot.webidl, components/script/dom/range.rs, components/script/dom/htmlscriptelement.rs and 50 more
  • @KiChjang: components/script/dom/htmltextareaelement.rs, components/script/dom/servoparser/mod.rs, components/script/dom/webidls/DocumentOrShadowRoot.webidl, components/script/dom/range.rs, components/script/dom/htmlscriptelement.rs and 50 more
  • @paulrouget: ports/servo/resources.rs
  • @emilio: components/style/stylesheet_set.rs, components/style/author_styles.rs, components/layout/construct.rs, components/layout/query.rs
@highfive

This comment has been minimized.

Copy link

commented Apr 16, 2019

warning Warning warning

  • These commits include an empty title element (<title></title>). Consider adding appropriate metadata.
  • These commits modify unsafe code. Please review it carefully!

@ferjm ferjm force-pushed the ferjm:media.ui branch from d078606 to be65763 Apr 18, 2019

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented Apr 23, 2019

Do you have an example available that I can use to diagnose the layout problems with the media controls?

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented Apr 23, 2019

In general, try absolute positioning if you want something that is guaranteed to work right. Trying to lay out with inlines is annoying and error-prone.

@ferjm

This comment has been minimized.

Copy link
Member Author

commented Apr 24, 2019

Do you have an example available that I can use to diagnose the layout problems with the media controls?

You should be able to see the media controls in action with this test running Servo with the dom.shadowdom.enabled pref on:

./mach run -d -- --pref dom.shadowdom.enabled https://ferjm.github.io/web-api-tests/video/controls.html

Thanks in advance for your help!

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented Apr 25, 2019

☔️ The latest upstream changes (presumably #23263) made this pull request unmergeable. Please resolve the merge conflicts.

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented Apr 27, 2019

OK. This needs a rebase.

@ferjm ferjm force-pushed the ferjm:media.ui branch 2 times, most recently from 367c155 to 73dccc2 Apr 30, 2019

@ferjm ferjm removed the S-needs-rebase label Apr 30, 2019


// Servo only API to get an instance of the controls of a specific
// media element matching the given id.
fn ServoGetMediaControls(&self, id: DOMString) -> Fallible<DomRoot<ShadowRoot>> {

This comment has been minimized.

Copy link
@Manishearth

Manishearth Apr 30, 2019

Member

Is there a danger of this getting overwritten by client code? We may need something like XOWs to do this right?

(Not that we have to do that in this PR but we should be careful when shipping)

This comment has been minimized.

Copy link
@ferjm

ferjm May 13, 2019

Author Member

I guess client code code could overwrite this. But I cannot think of a way to exploit that in a dangerous way. In any case, I agree that we may need something like XOWs to do this right.

@jdm

This comment has been minimized.

Copy link
Member

commented May 1, 2019

16:41 <pcwalton> jdm: it looks like the issue is really a shadow DOM issue and not so much a layout issue
16:41 <pcwalton> with the media controls
16:41 <jdm> oh?
16:41 <pcwalton> as far as I can tell, yes, because if you put the controls separately in the HTML file the layout works
16:42 <pcwalton> something about the shadow DOM makes it fail
16:42 <pcwalton> I can make a working version of the controls layout for you
16:42 <pcwalton> and you can figure out what's going on with the shadow DOM? I don't know the shadow DOM code at all
16:47 <jdm> sorting out the shadow dom sounds like the right choice
16:49 <jdm> pcwalton: do you have any insight into "The controls' div container ends up as sibbling of the media element in the flow tree while IIUC it should end up as a child."?
16:49 <pcwalton> that seems like a shadow DOM issue, yeah
16:50 <pcwalton> because if I make it not part of the shadow DOM then it works
@jdm

This comment has been minimized.

Copy link
Member

commented May 1, 2019

The best way forward might be to trace through building the flow tree for the two cases (part of the shadow DOM and not) and finding where the behaviour differs?

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented May 1, 2019

Video is replaced content, so you should not be trying to hack layout to render anything underneath <video>. Notice that if you write <video>hello</video> in Firefox, "hello" is not rendered.

Instead you should use the shadow DOM to replace <video/> with something like <div id=video-container><video/><div id=controls>...</div></div>.

Show resolved Hide resolved resources/media_controls.css Outdated
@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 2, 2019

Thanks for looking into this @pcwalton

Video is replaced content, so you should not be trying to hack layout to render anything underneath <video>.

Yes, I understand that. The temporary hack is only there to allow me to see the controls while coding them.

Notice that if you write <video>hello</video> in Firefox, "hello" is not rendered.

Indeed.

FWIW in Firefox the DOM looks like this for a <video> element with controls:

Screenshot 2019-05-02 at 10 00 07

The <video> element has a shadow root child where the media controls container is appended to.

I don't understand Gecko's layout at all (even less than Servo's layout), but it may seem that Gecko is treating <video> elements as replaced content but they are not considered as leafs?

Instead you should use the shadow DOM to replace <video/> with something like <div id=video-container><video/><div id=controls>...</div></div>.

Sorry, I am not sure I understand what you propose here. Would this replacement happen in the actual DOM tree?

The sole point of internally wrapping the controls in a closed shadow tree is to encapsulate and hide them from content.

Also, I believe wrapping the <video> element with a <div> would break scripts in situations like:

<div id="author-video-container">
  <video></video>
</div>
const video = document.getElementById("author-video-container").firstChild;
// Here video would be `<div id=video-container>` instead of `<video>` as the author would expect.
@emilio

This comment has been minimized.

Copy link
Member

commented May 2, 2019

So the Gecko setup here is that <video> is a replaced box for the purpose of sizing, but not a "leaf", which basically means "this box can have no other boxes inserted into it", and the content replacement happens using Shadow DOM.

But I'm not sure how that's quite relevant to this. Servo doesn't have the concept of "leaf" / "not-leaf", and the checks @ferjm is adding to the flow constructor are just special-casing <video> to render the children(), which since there's shadow DOM, it's not the DOM kids, but the replaced content you insert in the ShadowRoot.

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 2, 2019

16:49 pcwalton: do you have any insight into "The controls' div container ends up as sibbling of the media element in the flow tree while IIUC it should end up as a child."?
16:49 that seems like a shadow DOM issue, yeah
16:50 because if I make it not part of the shadow DOM then it works

FWIW something like

<div>
  #shadow-root 
    <div>
       ...
    </div>
</div>

works fine, the content of the <div> is properly rendered.

While something like:

<video>
  #shadow-root 
    <div>
      ...
    </div>
</video>

Does not work. The <div> is not rendered (without cb0ab4d)

So I suspect the problem is not Shadow DOM, but the layout of media elements.

@emilio

This comment has been minimized.

Copy link
Member

commented May 2, 2019

I think you may be talking past each other a bit... So what @ferjm is doing looks right (as in, correct modulo the pseudo-element bit, albeit hacky) to me, since children() does the right thing wrt Shadow DOM. The issue with the video controls may be that <video> is an inline by default, and you're stashing a block inside, and Servo's handling of IB splits is not great.

@emilio

This comment has been minimized.

Copy link
Member

commented May 2, 2019

That being said I may be wrong too, does this patch make the controls have the right layout tree if <video> is display: block?

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 2, 2019

I am afraid there is no change in the layout tree if <video> is display:block.

A test like

<html lang="en">
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Video test</title>
    <style>
      video {
        display: block;
      }
    </style>
  </head>
  <body>
    <video src="https://addpipe.com/sample_vid/short.mp4" poster="https://addpipe.com/sample_vid/poster.png"></video>
    <script>
      setTimeout(() => {
        document.querySelector("video").setAttribute("controls", "");
      });
    </script>
  </body>
</html>

Produces a DOM tree like:

 <html> (0x12ce91160)
    <head> (0x12ce911c0)
      <text node> (0x12ce911f0)
      <meta> (0x12ce91220)
      <text node> (0x12ce912b0)
      <title> (0x12ce912e0)
        <text node> (0x12ce91310)
      <text node> (0x12ce91340)
      <style> (0x12ce91370)
        <text node> (0x12ce913a0)
      <text node> (0x12ce913d0)
    <text node> (0x12ce91400)
    <body> (0x12ce91430)
      <text node> (0x12ce91460)
      <video> (0x12ce91490)
        <script> (0x12ce916d0)
          <text node> (0x12ce91700)
        <div> (0x12ce91730)
          <text node> (0x12ce917f0)
          <button id=play-pause-button> (0x12ce91820)
          <text node> (0x12ce91880)
          <input id=progress> (0x12ce918b0)
          <text node> (0x12ce91a00)
          <span id=position-duration-box> (0x12ce91a30)
            <text node> (0x12ce91ac0)
            <span id=durationSpan> (0x12ce91af0)
              <text node> (0x12cec8670)
            <text node> (0x12ce91b80)
          <text node> (0x12ce91bb0)
          <button id=volume-switch> (0x12ce91be0)
          <text node> (0x12ce91c40)
          <input id=volume-level> (0x12ce91c70)
          <text node> (0x12ce91dc0)
        <script> (0x12ce91f70)
          <text node> (0x12ce91fa0)
      <text node> (0x12ce91520)
      <script> (0x12ce91550)
        <text node> (0x12ce91580)
      <text node> (0x12ce915b0)

And a flow tree like:

┌ Post layout flow tree
│  ├─ Block(124588710)
│  │         sc=StackingContextId(0)
│  │         pos=LogicalRect(H LTR, i1024px×b740px, @ (i0px,b0px))
│  │         floatspec-in=L 0px R 0px
│  │         floatspec-out=L 0px R 0px
│  │         overflow=Overflow { scroll: TypedRect(1050px×740px at (0px,0px)), paint: TypedRect(1050px×740px at (0px,0px)) }
│  │         children=1
│  │         abs-descendents=1
│  │  ├─ ↑↑ Fragment for block:
│  │  │         SpecificFragmentInfo::Generic(97) []
│  │  │         border_box=LogicalRect(H LTR, i1024px×b740px, @ (i0px,b0px))
│  │  ├─ Block(1244ef390)
│  │  │         sc=StackingContextId(0)
│  │  │         pos=LogicalRect(H LTR, i1024px×b289px, @ (i0px,b8px))
│  │  │         floatspec-in=L 0px R 0px
│  │  │         floatspec-out=L 0px R 0px
│  │  │         overflow=Overflow { scroll: TypedRect(1042px×289px at (8px,0px)), paint: TypedRect(1042px×289px at (8px,0px)) }
│  │  │         children=1
│  │  │  ├─ ↑↑ Fragment for block:
│  │  │  │         SpecificFragmentInfo::Generic(93) []
│  │  │  │         border_box=LogicalRect(H LTR, i1008px×b289px, @ (i8px,b0px))
│  │  │  │         margin=LogicalMargin(H LTR, i:8px..8px b:8px..8px)
│  │  │  ├─ Block(124588490)
│  │  │  │         sc=StackingContextId(0)
│  │  │  │         pos=LogicalRect(H LTR, i1008px×b289px, @ (i8px,b0px))
│  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │         overflow=Overflow { scroll: TypedRect(1042px×289px at (0px,0px)), paint: TypedRect(1042px×289px at (0px,0px)) }
│  │  │  │         children=1
│  │  │  │  ├─ ↑↑ Fragment for block:
│  │  │  │  │         SpecificFragmentInfo::Media(92) []
│  │  │  │  │         border_box=LogicalRect(H LTR, i570px×b289px, @ (i0px,b0px))
│  │  │  │  │         margin=LogicalMargin(H LTR, i:0px..438px b:0px..0px)
│  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  ├─ Block(124418790)
│  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │         pos=LogicalRect(H LTR, i1042px×b30.133333333333333px, @ (i0px,b0px))
│  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │         overflow=Overflow { scroll: TypedRect(1042px×30.133333333333333px at (0px,0px)), paint: TypedRect(1042px×30.133333333333333px at (0px,0px)) }
│  │  │  │  │         children=1
│  │  │  │  │  ├─ ↑↑ Fragment for block:
│  │  │  │  │  │         SpecificFragmentInfo::Generic(78) []
│  │  │  │  │  │         border_box=LogicalRect(H LTR, i1042px×b30.133333333333333px, @ (i0px,b0px))
│  │  │  │  │  │         border_padding=LogicalMargin(H LTR, i:9px..9px b:0px..0px)
│  │  │  │  │  ├─ Inline(1244b7950)
│  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │         pos=LogicalRect(H LTR, i1024px×b30.133333333333333px, @ (i9px,b0px))
│  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(180.35px×29.133333333333333px at (0px,0px)), paint: TypedRect(180.35px×29.133333333333333px at (0px,0px)) }
│  │  │  │  │  │         children=4
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::InlineBlock(69) []
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i24px×b24px, @ (i0px,b2px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(80) [" "]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i4px×b16px, @ (i24px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::InlineBlock(72) []
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i16.083333333333332px×b26px, @ (i28px,b0px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(81) [" "]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i4px×b16px, @ (i44.083333333333336px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(75) ["0:00 "]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i32.45px×b16px, @ (i48.083333333333336px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(76) ["/ 2:26"]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i36.9px×b16px, @ (i80.53333333333333px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(77) [" "]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i4px×b16px, @ (i117.43333333333334px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::InlineBlock(74) []
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i24px×b24px, @ (i121.43333333333334px,b2px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(83) [" "]
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i4px×b16px, @ (i145.43333333333334px,b13.133333333333333px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─
│  │  │  │  │  │  │         SpecificFragmentInfo::InlineBlock(16) []
│  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i30.916666666666668px×b26px, @ (i149.43333333333334px,b0px))
│  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─ Block(124589390)
│  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │         pos=LogicalRect(H LTR, i24px×b24px, @ (i0px,b0px))
│  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(24px×24px at (0px,0px)), paint: TypedRect(24px×24px at (0px,0px)) }
│  │  │  │  │  │  │         damage=REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  └─ ↑↑ Fragment for block:
│  │  │  │  │  │  │  │         SpecificFragmentInfo::Generic(68) []
│  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i24px×b24px, @ (i0px,b0px))
│  │  │  │  │  │  │  │         border_padding=LogicalMargin(H LTR, i:3.3333333333333335px..3.3333333333333335px b:0px..0px)
│  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW | REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  ├─ Block(124586410)
│  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │         pos=LogicalRect(H LTR, i16.083333333333332px×b26px, @ (i0px,b0px))
│  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(16.083333333333332px×26px at (0px,0px)), paint: TypedRect(16.083333333333332px×26px at (0px,0px)) }
│  │  │  │  │  │  │         children=1
│  │  │  │  │  │  │         damage=REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  ├─ ↑↑ Fragment for block:
│  │  │  │  │  │  │  │         SpecificFragmentInfo::Generic(70) []
│  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i16.083333333333332px×b26px, @ (i0px,b0px))
│  │  │  │  │  │  │  │         border_padding=LogicalMargin(H LTR, i:4.333333333333333px..4.333333333333333px b:1px..1px)
│  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW | REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  ├─ Inline(1244b6450)
│  │  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │  │         pos=LogicalRect(H LTR, i7.416666666666667px×b24px, @ (i4.333333333333333px,b1px))
│  │  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(7.416666666666667px×13.333333333333334px at (0px,5.333333333333333px)), paint: TypedRect(7.416666666666667px×13.333333333333334px at (0px,5.333333333333333px)) }
│  │  │  │  │  │  │  │  └─
│  │  │  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(71) ["0"]
│  │  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i7.416666666666667px×b13.333333333333334px, @ (i0px,b5.333333333333333px))
│  │  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
│  │  │  │  │  │  ├─ Block(124589110)
│  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │         pos=LogicalRect(H LTR, i24px×b24px, @ (i0px,b0px))
│  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(24px×24px at (0px,0px)), paint: TypedRect(24px×24px at (0px,0px)) }
│  │  │  │  │  │  │         damage=REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  └─ ↑↑ Fragment for block:
│  │  │  │  │  │  │  │         SpecificFragmentInfo::Generic(73) []
│  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i24px×b24px, @ (i0px,b0px))
│  │  │  │  │  │  │  │         border_padding=LogicalMargin(H LTR, i:3.3333333333333335px..3.3333333333333335px b:0px..0px)
│  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW | REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  ├─ Block(124586b90)
│  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │         pos=LogicalRect(H LTR, i30.916666666666668px×b26px, @ (i0px,b0px))
│  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(30.916666666666668px×26px at (0px,0px)), paint: TypedRect(30.916666666666668px×26px at (0px,0px)) }
│  │  │  │  │  │  │         children=1
│  │  │  │  │  │  │         damage=REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  ├─ ↑↑ Fragment for block:
│  │  │  │  │  │  │  │         SpecificFragmentInfo::Generic(14) []
│  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i30.916666666666668px×b26px, @ (i0px,b0px))
│  │  │  │  │  │  │  │         border_padding=LogicalMargin(H LTR, i:4.333333333333333px..4.333333333333333px b:1px..1px)
│  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW | REFLOW_OUT_OF_FLOW | REFLOW
│  │  │  │  │  │  │  ├─ Inline(1244b5490)
│  │  │  │  │  │  │  │         sc=StackingContextId(1)
│  │  │  │  │  │  │  │         pos=LogicalRect(H LTR, i22.25px×b24px, @ (i4.333333333333333px,b1px))
│  │  │  │  │  │  │  │         floatspec-in=L 0px R 0px
│  │  │  │  │  │  │  │         floatspec-out=L 0px R 0px
│  │  │  │  │  │  │  │         overflow=Overflow { scroll: TypedRect(22.25px×13.333333333333334px at (0px,5.333333333333333px)), paint: TypedRect(22.25px×13.333333333333334px at (0px,5.333333333333333px)) }
│  │  │  │  │  │  │  │  └─
│  │  │  │  │  │  │  │  │         SpecificFragmentInfo::ScannedText(15) ["100"]
│  │  │  │  │  │  │  │  │         border_box=LogicalRect(H LTR, i22.25px×b13.333333333333334px, @ (i0px,b5.333333333333333px))
│  │  │  │  │  │  │  │  │         damage=REPOSITION | STORE_OVERFLOW
@pcwalton

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

Seems to me that this issue is clearly shadow DOM specific, because if you render the controls without using shadow DOM (by pasting them directly into the HTML) then they are laid out properly below the video. So it has something to do with the implementation of shadow DOM in Servo, which I know nothing about.

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

One random hypothesis: Shadow DOM could be failing to invalidate parts of the DOM properly.

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 2, 2019

because if you render the controls without using shadow DOM (by pasting them directly into the HTML) then they are laid out properly below the video

Are you pasting them as children of the <video> element?

@pcwalton

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

The flow/fragment distinction means that Servo layout just doesn't have the concept of a replaced content box that can contain other elements (that are rendered). The spec says that replaced content can't contain visible elements, and Servo layout is designed around this.

The only way it could possibly work is to treat video as a "background" of a Generic fragment, like for CSS image backgrounds. But this would be a horrible hack.

@ferjm ferjm force-pushed the ferjm:media.ui branch from e5aeab7 to 61132de May 14, 2019

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

@bors-servo try=wpt

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

⌛️ Trying commit 61132de with merge 5db53dd...

bors-servo added a commit that referenced this pull request May 14, 2019

Auto merge of #23208 - ferjm:media.ui, r=<try>
Media controls

<strike>This is still highly WIP. It depends on #22743 and so it is based on top of it.

The basic controls functionality is there, but the layout is highly broken. There is a hack to at least make the controls render on top of the video, but it is not correctly positioned. The controls' div container ends up as sibbling of the media element in the flow tree while IIUC it should end up as a child.</strike>

- [X] `./mach build -d` does not report any errors
- [ ] `./mach test-tidy` does not report any errors
- [X] These changes fix #22721 and #22720

There is at least an extra dependency to improve the functionality and visual aspect: #22728.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23208)
<!-- Reviewable:end -->
@bors-servo

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

☀️ Test successful - linux-rel-css, linux-rel-wpt, status-taskcluster
State: approved= try=True

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

@highfive highfive assigned emilio and unassigned asajeffrey May 14, 2019

@emilio
Copy link
Member

left a comment

I don't understand why you need the IS_USER_AGENT_WIDGET flag spread all around the DOM tree.

If you have a UA-widget shadow root (i.e., video controls), you shouldn't get light tree elements in the first place.

If you don't, then you should treat it as a regular replaced element and such, but that flag is not the way. You could just look at the media element's shadow root, right?

Show resolved Hide resolved components/layout/construct.rs Outdated
Show resolved Hide resolved components/layout/construct.rs Outdated
if node.type_id() == Some(LayoutNodeType::Element(LayoutElementType::HTMLMediaElement))
{
// Do not treat media elements as leafs.
self.build_flow_for_block(node, None)

This comment has been minimized.

Copy link
@emilio

emilio May 14, 2019

Member

This is wrong. data:text/html,a <video controls style="display: inline"></video> b should be in the same line, so <video> shouldn't be a block.

This comment has been minimized.

Copy link
@ferjm

ferjm May 16, 2019

Author Member

I ended up reverting this change. I couldn't make it work for inline layout :(. I'll file a follow up to fix this.

Show resolved Hide resolved components/script/dom/node.rs Outdated

@ferjm ferjm force-pushed the ferjm:media.ui branch 2 times, most recently from eb55938 to 847fe51 May 15, 2019

@ferjm

This comment has been minimized.

Copy link
Member Author

commented May 16, 2019

r? @emilio

@ferjm ferjm force-pushed the ferjm:media.ui branch from 0a95333 to cdea2cd May 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.