New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Declarative Shadow DOM #510

Closed
tabatkins opened this Issue Sep 12, 2017 · 134 comments

Comments

@tabatkins
Contributor

tabatkins commented Sep 12, 2017

One of the promises of early Shadow DOM (and several other Parkour-descended products) was that while we would build the first version as an imperative (JS-based) API, for flexibility and composability, we'd come in later and backfill a declarative version that made it very easy to use for common cases. We haven't done this yet for Shadow DOM, and the spec is stable enough now that I think we can safely do it.


Rough proposal:

<div-or-something>
  <template attach-shadow shadow-mode="open | closed">
    ...shadow content...
  </template>
</div-or-something>

That is, using a <template> element with an attach-shadow attribute automatically (during parsing) attaches its contents to its parent element as a shadow root. The mode is selected by the optional shadow-mode attribute; if omitted, it defaults to "open" (or we can make it required, like the JS API does?).


Reasoning:

Today you can get something close to declarative with JS like:

<parent-element>
</parent-element>
<script>
"use strict";

const shadowRoot = document.currentScript.previousElementSibling.attachShadow({ mode: "open" });
shadowRoot.innerHTML = `<child-element></child-element>`;
</script>

This has several downsides:

  1. Doesn't work nicely with syntax highlighting.
  2. Doesn't work nicely with any tooling that wants to be able to output HTML.
  3. Doesn't nest easily. (That is, having an element in the shadow contain another declarative shadow.)
  4. Can't include <script> in the shadow unless you remember to do the various tricks to avoid having a literal </script> in the string.
  5. Also have to be aware of which quoting character you're using, and remember to escape it when you're writing your page.
  6. Including user-generated content is now more complicated; it not only needs to be escaped as safe HTML, but as safe script and string contents, too, which are decently more complex (and not, afaik, supported by the core libraries in PHP or Python).
  7. Lots of non-trivial typing for something intended to be easy and widespread.
  8. Inline script isn't compatible with safe CSP practice, unless you go the extra mile to add nonces (more effort! must use crypto safely!)
  9. Inline script isn't run for .mhtml archives (at least in Chrome, for historical reasons).
  10. Users with JS turned off won't get any page at all, for no good reason - page might have just been wanting style isolation, but is now tied to JS evaluation.
  11. Inline script halts the speculative parser, I think?
  12. Doesn't allow for server-side rendering (doing as much of the page as possible on the server, only using client-side JS for final fixups and event hookups, etc). The page instead has to do all this work on client-side (which is apparently fairly slow, per #510 (comment))
@domenic

This comment has been minimized.

Member

domenic commented Sep 13, 2017

I am not personally convinced of the need here (but I don't oppose it). Let me try to help with some details that could be problematic if people want to advance this.

  • We don't use hyphens in built-in attribute names. So attachshadow/shadowmode instead of attach-shadow/shadow-mode
  • It seems like these could be combined into a single attribute, e.g. shadow="open|closed". If shadow is not present then it's a normal template.
  • We'll have to recapitulate the great defaulting wars that led to mode being mandatory in the JS API. In particular, we need to answer what <template shadow="asdf"> and <template shadow> do. Maybe the answer is nothing (as if you'd omitted the attribute), but at least for <template shadow> that seems unfortunate.
  • Harder question: how does this interact with the existing processing model for templates?
    • Template contents are parsed in a completely separate document, the template contents owner document. That prevents e.g. script from running or images from downloading. It sounds like that will no longer be the case here?
    • What does templateEl.content return? Maybe it becomes an alias for templateEl.shadowRoot? Is that too weird?
    • What are the cloning and adopting steps?
@annevk

This comment has been minimized.

Member

annevk commented Sep 13, 2017

I think I'd prefer <shadowroot mode="open|closed"> with the parser simply not treating this as a ShadowRoot insertion request if the mode attribute cannot be validated (until we can agree on a default).

This kind of feature also helps with first load where the server has done all the templating work to make rendering happen faster.

@domenic

This comment has been minimized.

Member

domenic commented Sep 13, 2017

That would require more parser work and thus ecosystem churn though, right? On the same order of magnitude as that when we originally introduced template? Otherwise I agree it'd be nicer.

@annevk

This comment has been minimized.

Member

annevk commented Sep 13, 2017

It should be less work now we have <template>, but probably still non-trivial. I'm not a big fan of overloading elements, that hasn't worked well historically (e.g., <object> and <embed> are still a poorly understood mess).

@annevk

This comment has been minimized.

Member

annevk commented Sep 13, 2017

(Note that I suspect that overloading <template> would also require parser changes, due to the things you mentioned.)

@treshugart

This comment has been minimized.

treshugart commented Sep 13, 2017

Hi! It's me again. Sorry. Some background:

I've tried to keep most of the history and rationale for our thoughts in the README of the skatejs/ssr repo, but some of it might be missing. It's worth a study to at least see where we're coming from.

The thing I've found most interesting in this work is that the imperative API is exclusive in its DOM behaviour. For example, <div>test</div> means something completely different when attachShadow() is imperatively invoked. Finding a declarative equivalent that doesn't break existing assumptions is difficult. For example:

<div>
  World
  <shadow-root>
    Hello, <slot />!
  </shadow-root>
<div>

Whether or not this is used with a <template /> element or something else, there's several problems with this:

  1. SEO and content order when read in engines that don't execute JS.
  2. What happens to <shadow-root />? Does it get removed?
  3. What happens to childNodes? Attaching a root modifies this.
  4. Does appendChild(document.createElement('shadow-root')) work?

Libraries like React (and many more) have based themselves on the pillar that a core subset of DOM works as intended. If appending a shadow root mutates the host and makes childeNodes behave differently, this breaks the web. Inversely, if this is only available to the initial parse, it seems inconsistent. Meaning if <div><shadow-root /></div> is only available to the parser, but means nothing when using the imperative alternatives (i.e. appendChild(shadowRootElement)), then expectations of declarative vs imperative APIs are inconsistent.

I'm sorry if my thoughts seem a bit disjointed right now. Somebody linked this to me and it's a bit late here, but I really wanted to lay out some of my thoughts otherwise I may have forgotten about it (I do web components in my spare time).

I'm excited to see traction on this!

@matthewp

This comment has been minimized.

matthewp commented Sep 13, 2017

Thanks for starting this issue @tabatkins. To add some weight that this is a need, I tried the JS approach that you mentioned some time ago and it was fairly slow: https://discourse.wicg.io/t/declarative-shadow-dom/1904/8

@tabatkins

This comment has been minimized.

Contributor

tabatkins commented Sep 13, 2017

We don't use hyphens in built-in attribute names. So attachshadow/shadowmode instead of attach-shadow/shadow-mode

Ah yeah, right.

It seems like these could be combined into a single attribute, e.g. shadow="open|closed". If shadow is not present then it's a normal template.

I did that at first, but separated it because the mode isn't a positional attribute of attachShadow, it's just a required entry in the options dict. I thought we might add more in the future, but now that I'm thinking, I guess we can't add more required options anyway as it would break the API. So sure, we can combine them.

Maybe the answer is nothing (as if you'd omitted the attribute), but at least for <template shadow> that seems unfortunate.

Strongly agree. I think there's an obvious answer - default to open, because you're just using this as a styling boundary; nothing more special is happening at all anyway, because no JS.

Template contents are parsed in a completely separate document, the template contents owner document. That prevents e.g. script from running or images from downloading. It sounds like that will no longer be the case here?

My intention is that it'll append the template contents to the auto-created shadow root. I think scripts'll run at that point, yeah?

What does templateEl.content return? Maybe it becomes an alias for templateEl.shadowRoot? Is that too weird?

Per the above, I guess the append automatically removes them from the template's doc fragment? The answer is then ".content is empty because its contents have been transferred out". Or is there a stamping operation that doesn't remove the contents? I'm not sure of the details here. I'm flexible, whatever answer falls out is fine.

What are the cloning and adopting steps?

I think in my conception this is a parser operation more or less, so cloning/adopting just does whatever those operations do for elements that have shadow roots attached in the JS way. I'm not sure what the implications of that are, tho.


This kind of feature also helps with first load where the server has done all the templating work to make rendering happen faster.

Ah yes, thank you, that's another tick in the Pro column.


[Adding a <shadowroot> element] would require more parser work and thus ecosystem churn though, right? On the same order of magnitude as that when we originally introduced template? Otherwise I agree it'd be nicer.

I completely agree - it would definitely be nicer, but given the non-trivial parser work, I was thinking leaning on template would be better. On the other hand, I guess now that template exists widely, one can lean on that for polyfilling (doing <template><shadowroot>...) and we could add a new element. I don't have a strong opinion on this, because I understand the tradeoffs. Whatever works best for everyone.


SEO and content order when read in engines that don't execute JS.

There's no JS here. Tools that read the DOM would have to update to know about declarative shadow roots, sure, but that's vastly easier than supporting JS.

What happens to <shadow-root />? Does it get removed?

I'd think it would just stay there, like template does. The shadow-attaching is a parse-time operation, it's a dead element afterwards.

What happens to childNodes? Attaching a root modifies this.

What do you mean? The element's childNodes are unaffected by the presence of a shadow root. The children get distributed, which matters for CSS and some other stuff down-stream, but the DOM doesn't care at all.

Does appendChild(document.createElement('shadow-root')) work?

No, this is a parse-time operation. Post-parsing, the shadowroot element is dead, identical to a plain template. If you have JS, just use attachShadow(). ^_^

@robdodson

This comment has been minimized.

robdodson commented Sep 13, 2017

The answer is then ".content is empty because its contents have been transferred out". Or is there a stamping operation that doesn't remove the contents?

I think template.content.cloneNode leaves the content in place (sorry on my phone right now so can't check).

Really excited to see this discussion happening! @kevinpschaaf @samuelli

@bedeoverend

This comment has been minimized.

bedeoverend commented Sep 13, 2017

There's no JS here. Tools that read the DOM would have to update to know about declarative shadow roots, sure, but that's vastly easier than supporting JS.

This for me is a potential issue - if we're relying on support from SEO engines / HTML scrapers to add support, it might hurt adoption as people are after a solution with current bots / tools - particularly given the ambiguity around what bots can do already.

Is this a concern for others? If so, it might be worth exploring a declarative 'composed' DOM, rather than declarative Shadow DOM, so current bots can 'see' what the user will see. Something we're exploring over at skatejs/ssr

Apologies if this is sidetracking from the main issue, but feel its worth mentioning. Thanks for getting this discussion going!

@tabatkins

This comment has been minimized.

Contributor

tabatkins commented Sep 13, 2017

This for me is a potential issue - if we're relying on support from SEO engines / HTML scrapers to add support, it might hurt adoption as people are after a solution with current bots / tools - particularly given the ambiguity around what bots can do already.

That's a fully general counter-argument against any addition to HTML at all, and so can't really be used as a specific objection without more details.

If so, it might be worth exploring a declarative 'composed' DOM,

That's just... DOM, right? Like, normal ordinary DOM children.

@bedeoverend

This comment has been minimized.

bedeoverend commented Sep 13, 2017

That's a fully general counter-argument against any addition to HTML at all, and so can't really be used as a specific objection without more details.

Yeah that's fair - I'll go into specific details and whether tradeoffs are worthwhile.

That's just... DOM, right? Like, normal ordinary DOM children.

What I meant by this was declarative way to present the resultant composed tree after shadow root is attached and light DOM nodes are distributed - apologies if I'm getting the terminology wrong here.

More concretely, what I was thinking was a composed flag to tell the parser to pull a part this elements DOM into Shadow and Light DOMs.

<div composed>
  Hello <slot><strong>World</strong></slot>
</div>

which the parser would turn into:

<div>
  #shadow-root
    Hello <slot></slot>
<strong>World</strong>
</div>

This wouldn't be a 1:1 declarative version of imperative Shadow DOM API, just a way to tell the parser to construct an element's Shadow and Light trees. Straight off - and I'm sure there's a lot more I'm not considering - downsides are:

  • No support for undistributed light DOM nodes
  • No support for default slot content
  • Certainly a confusing look at declarative Shadow DOM

I realise this may be way off base, but I just wanted to throw it out there as an idea, incase supporting current bots is worth the tradeoffs.

@treshugart

This comment has been minimized.

treshugart commented Sep 13, 2017

That's just... DOM, right? Like, normal ordinary DOM children.

Yes, but this is useful. People outside of the WC community actually want this.

What we're proposing is two distinct modes:

  1. Shadow DOM and CSS encapsulation via attachShadow() (the current state of things)
  2. Only CSS encapsulation via the composed attribute (maybe also works via attachShadow({ composed: true }) but that can be worked out later. Declarative design first IMO).

To concretely illustrate this, React is a good example.

class Hello extends Component {
  static defaultProps = {
    name: 'World'
  }
  render () => (
    <div composed>
      <style>
        /* Works on <div composed>. */
        :host {}

        /* This works because of <slot />! */
        ::slotted(*) {}

        /* Only works for top-level span, not the slotted one. */
        span { font-weight: bold; }
      </style>
      Hello,{' '}
      {/* This span would be affected... */}
      <span>
        <slot>
          {/* ...but not this one. */}
          <span>{ this.props.name }</span>
        </slot>
      </span>!
    </div>
  )
}

The idea here is a lot like the old <style scoped /> form of encapsulation, but it differs because the full gamut of Shadow CSS works (:host, ::slotted(), etc).

The <slot /> element provides an encapsulation barrier that CSS selectors cannot cross but that's it (querySelector() still works). The assignedNodes() method does not return anything because nothing is being projected. In this mode, DOM accessors are not encapsulated. However, to turn this into full Shadow DOM, one can call attachShadow() and the composed tree is reverse engineered:

  1. The host receives a shadow root.
  2. childNodes of slots are attached to the host, thus are projected.
  3. Any content within a slot that has the default attribute (flagging for default content) is not attached to the host, and remains as default content.
  4. For named slots, this is declared as normal and is thus projected as normal.

This is literally the algorithm we're employing for declarative shadow DOM at the moment. The thing that's nice about it is that it's backward compatible and it fits very closely with the current model for shadow roots. @bedeoverend did I miss anything in that list?

EDIT

My proposal and @bedeoverend's proposal are slightly different. In his, he states that composed should reverse engineer the DOM tree whereas in mine it leaves it alone until attachShadow() is called.

@treshugart

This comment has been minimized.

treshugart commented Sep 14, 2017

The "laugh" here along with the comment (not the comment alone) seems a bit condescending. We're outsiders of the W3C desperately trying to have our say in context of our extensive experience in working with these APIs and being in close working contact with other communities. Though, subtle, this downplays that and makes outsiders less motivated to participate which is bad for the specs. Can we please have a discussion here that places a bit more weight on non-W3C members' experience? Thanks!

screen shot 2017-09-14 at 10 04 47 am

@bedeoverend

This comment has been minimized.

bedeoverend commented Sep 14, 2017

@bedeoverend did I miss anything in that list?

Don't think so - only thing is we're looking to handle undistributed nodes, but IMO that shouldn't be baked into platform.

FWIW I'm still unsure about whether composed should be an entirely different mode, or whether it should just trigger the HTML parser to behave differently while parsing that element. I'd be for a a different 'CSS-only' mode, but is it reaching too far right now? Perhaps there's an iterative way to get there? Keen to balance what's feasible now and what's going to garner adoption from wider web community.

@bedeoverend

This comment has been minimized.

bedeoverend commented Sep 14, 2017

Though, subtle, this downplays that and makes outsiders less motivated to participate which is bad for the specs. Can we please have a discussion here that places a bit more weight on non-W3C members experience?

I agree with this. I appreciate it must be incredibly hard to work on specs and and the same time keep it open, but reactions which minimise external contribution only serve to ostracise the wider community. I want to try help, and I'm happy to be told my ideas aren't going to work, or I'm not communicating them well enough, but there's got to be a better way of doing that while keeping the community open.

@treshugart

This comment has been minimized.

treshugart commented Sep 14, 2017

FWIW I'm still unsure about whether composed should be an entirely different mode, or whether it should just trigger the HTML parser to behave differently while parsing that element. I'd be for a a different 'CSS-only' mode, but is it reaching too far right now? Perhaps there's an iterative way to get there? Keen to balance what's feasible now and what's going to garner adoption from wider web community.

I've had discussions about this with several outside of the WC community. Since they have their own component model (whether or not they should be using Custom Elements is besides the point), they don't need the DOM encapsulation, they just want the CSS aspect. If what their internals expect changes underneath them (their declared DOM changes from what they think it looks like) then it precludes them from declarative usage. This is a barrier to entry for them because they must modify their internals.

I truly believe what I've proposed actually fits as two modes that can be composed together to achieve what you want here. It can service both use-cases while maintaining compatibility with libraries and frameworks, without requiring they update their internals.

@dominiccooney

This comment has been minimized.

dominiccooney commented Sep 14, 2017

@annevk wrote:

I think I'd prefer <shadowroot ...

Using <template ...> would make this much easier to polyfill. For example one could write a custom element which does the parent node stuffing and the syntax would look like <template is="shadow-root" shadow="...

I guess it will be easier to add "popping the element stack and the element is a shadow-setting template" steps to the HTML parser than to separate out the context-inheriting stuff of the template to reuse it for shadowroot.

If the idea is to avoid the adoption tree walk by creating the nodes in the target document then, yeah, not complicating template is better. The inoperativeness of template parsing (specifically not running script and custom elements) would be tricky to preserve.

@hayatoito

This comment has been minimized.

Collaborator

hayatoito commented Sep 14, 2017

I tend to agree @domenic. I am not personally convinced of the need here, but I don't oppose it.

What happens to ? Does it get removed?

I'd think it would just stay there, like template does. The shadow-attaching is a parse-time operation, it's a dead element afterwards.

If it would just stay there as a dead element, the memory usage would increase, doubled or maybe more than that.

Suppose the following html, using declarative Shadow DOM via <shadowroot>,

<custom-a>
  <shadowroot>
    <slot></slot>
    <div></div>
  </shadowroot>
  <custom-b>
    <shadowroot>
      <slot></slot>
      <custom-c>
         <shadowroot>
            <slot></slot>
            <div></div>
         </shadowroot>
         <div></div>      
      </custom-c>
    </shadowroot>
  </custom-b>
</custom-b>

In this case, the composed tree which the rendering engine actually holds would be:

custom-a
  ::shadow-root
     slot
     div
  shadowroot
    slot
    div
  custom-b
    ::shadow-root
      slot
      custom-c
         ::shadow-root
            slot
            div
         shadowroot
            slot
            div
         div
    shadowroot
      slot
      custom-c
         shadowroot
            slot
            div
         div

I would prefer to remove a <shadowroot> element from the light tree after parsing and attaching it as a shadow tree, to save memory.

This can be:

custom-a
  ::shadow-root
     slot
     div
  custom-b
    ::shadow-root
      slot
      custom-c
         ::shadow-root
            slot
            div
@dominiccooney

This comment has been minimized.

dominiccooney commented Sep 14, 2017

I would prefer to remove ` (sic) element from the light tree after parsing, to save memory.

Removing it would also make feature detection easier.

It's probably not a big deal but implementations should be able to avoid creating the template element because you won't be able to observe its brief existence (unless it is a custom element.)

@sebmarkbage

This comment has been minimized.

sebmarkbage commented Sep 14, 2017

I'll echo @treshugart's point that, for React, the shadow DOM is only overhead and complexity, when the only thing we'd really want out of this is scoped CSS. If we can't have that, we could work with a shadow DOM too. I'm sure @wycats had thoughts on this too.

I would expect many trees to result in code like:

<div>
  <template attach-shadow shadow-mode="open">
    <div>
      <template attach-shadow shadow-mode="open">
        <div>
          <template attach-shadow shadow-mode="open">
            <div>
              <template attach-shadow shadow-mode="open">
                ...
              </template>
            </div>
          </template>
        </div>
      </template>
    </div>
  </template>
</div>

Since that's semantically what a modular composed tree would be like. We'd just treat the shadow tree as the new children tree. Anything else would be an optimization (which might need to bail out).

What would be very interesting, however, is if you could target a particular ID that was defined earlier to attach the shadow lazily later in the document stream.

<section>
  <h1>Heading</h1>
  <p id="content">Loading...</p>
   ...
</section>
...
<template attachto="content">
  Hello world!
</template>

Even then, I'd prefer it if it replaced the children with the content of template, but we could live with the workaround to use shadow DOM.

@slightlyoff

This comment has been minimized.

slightlyoff commented Sep 14, 2017

Given the extraordinary lift that was required to get <template> done (we had to get changes into XML's parsing behavior, e.g.), a new element for this seems like a non-starter.

@hayatoito

This comment has been minimized.

Collaborator

hayatoito commented Sep 14, 2017

From the perspective of implementation difficulty, whether <template newattribute> or <shadowroot>-ish new element shouldn't be a big deal, I guess, though it still needs non-trivial efforts.

From the perspective of spec, that might be a big deal.

@annevk

This comment has been minimized.

Member

annevk commented Sep 14, 2017

@treshugart I think that by definition declarative shadow trees would have to be somewhat new as shadow trees are new. You cannot expect it to map to appendChild() just like you cannot expect that for contents of <template>.

Now, having a separate kind of feature where there is some kind of boundary that ends up applying to CSS, but doesn't involve creation shadow trees, might be worthwhile, but I don't think it's a good idea to conflate that request with the topic of this issue: declarative shadow trees. There's not even an imperative API for what you're suggesting. I have seen the request before to decouple more and I can certainly appreciate that though I haven't seen a good enough solution yet. In any event, please open a separate issue to discuss that.

@hayatoito I think we'll hit about as much complexity either way and I suspect that overloading <template> this way will make it harder to add new features to it in the future.

@domenic

This comment has been minimized.

Member

domenic commented Sep 14, 2017

@treshugart I apologize, and have removed the offending emoji. I'll step back from this thread.

@tabatkins

This comment has been minimized.

Contributor

tabatkins commented Sep 14, 2017

I would prefer to remove a <shadowroot> element from the light tree after parsing and attaching it as a shadow tree, to save memory.

Sure, I don't have an opinion on this. If removing it makes more sense, let's remove it. That might end up being less confusing for authors too, since the element doesn't do anything if it's sitting there.

It's probably not a big deal but implementations should be able to avoid creating the template element because you won't be able to observe its brief existence (unless it is a custom element.)

Another reasonable argument for removing it.


I've had discussions about this with several outside of the WC community. Since they have their own component model (whether or not they should be using Custom Elements is besides the point), they don't need the DOM encapsulation, they just want the CSS aspect. If what their internals expect changes underneath them (their declared DOM changes from what they think it looks like) then it precludes them from declarative usage. This is a barrier to entry for them because they must modify their internals.

The current concept of CSS encapsulation is built on Shadow DOM. It can't be separated from it in a reasonable way; doing so would be a completely new, totally separate feature that would need its own justification. (And I can tell you right now, it would be a hard sell to justify having two totally different ways to encapsulate CSS.) I'm not willing to try and tie declarative shadow DOM to this sort of limitation. As Anne said, this should be addressed via a separate issue.

(I also, personally, am 100% on the "stop rolling your own component models, they're not interoperable and they lock users in, just use WC" train.)

@treshugart

This comment has been minimized.

treshugart commented Sep 14, 2017

Thank you, @domenic, and just to be clear - for posterity - you didn't necessarily "offend" me and it's not just you. This in isolation wouldn't be an issue. Community involvement in this working group has been pulling teeth for many, and this is one of the many forms of negativity (even if it's small) towards it. All we want is for our extensive experience to carry weight in the discussions.

I won't let this detract from the rest of the discussion. I'm over the moon this is even being discussed right now.

@hayatoito

This comment has been minimized.

Collaborator

hayatoito commented Sep 15, 2017

Sure, I don't have an opinion on this. If removing it makes more sense, let's remove it. That might end up being less confusing for authors too, since the element doesn't do anything if it's sitting there.

Regarding, template overloading (<template newattribute>) vs new element (<shadowroot>-ish),

if we can agree on "don't leave <template> element in the light tree", I prefer a new element, <shadowroot> or something, here because:

Descendant nodes of declarative shadow dom are effectively NOT inert, from user's perspective.
e.g.

  <custom-a>
    <template newattribute> (or <shadowroot>)
      <img src='...'>
      <script src='...'>
    </template>
  </custom-a>

In this case, <img> or <script> look inside of <template> element, but they would reside in the shadow tree, as the final result, after parsing html. We fetch resources for them. That sounds a big difference to me, as compared to usual <template> element's usage pattern. newattribute is too weak as a visual sign of this big different semantics.

In addition to that, as Dominic suggested, the engine (or parser) might want to optimize.

Instead of:

  1. Parse a template element (and its descendants), as usual
  2. Call attachShadow on the parent element of the template
  3. shadowRoot.appendChild(template.content);
  4. Remove template element from the light tree

We might want to optimize:

  1. Parser encounters opening <shadowroot>
  2. Call attachShadow on the parent element of the <shadowroot>
  3. Parser continues to parse descendants of <shadowroot>, and build a shadow tree directly
  4. Parser encounters a closing </shadowroot>.
  5. Parser continues to parse html, and continue to build a light tree

I am not sure how this approach is feasible, but that could be an option as a starter. If we choose this, there might be no longer a good reason to use <template>.

@justinfagnani

This comment has been minimized.

justinfagnani commented Sep 16, 2017

I can't speak to how difficult it would be to get a new element in, but @hayatoito's description of optimized <shadowroot> sounds perfectly aligned to the use-case, and not even an optimization, how how it should just work.

@hayatoito

This comment has been minimized.

Collaborator

hayatoito commented Mar 2, 2018

Google is interested in implementing the proposal. The general idea matches my expectation.

@rniwa

This comment has been minimized.

rniwa commented Mar 2, 2018

While we're generally in favor of introducing a mechanism to declaratively define a shadow root, we're STRONGLY against introducing a new parser macro. That would be a show stopper for us.

@annevk

This comment has been minimized.

Member

annevk commented Mar 2, 2018

@rniwa that sounds interesting. How would you reconcile the two, given that a ShadowRoot node is not an element?

@rniwa

This comment has been minimized.

rniwa commented Mar 2, 2018

For example, we can create a shadowroot element which attaches a shadow root on its parent and removes itself once all children had been inserted by the parser.

Introducing a new parser macro is like introducing a new synchronous DOM mutation event to us. It's simply not happening.

@annevk

This comment has been minimized.

Member

annevk commented Mar 3, 2018

That sounds very similar to a parser macro of sorts though. The only difference is that now you have to deal with script element children moving that element around and such.

@treshugart

This comment has been minimized.

treshugart commented Mar 4, 2018

It might be good to discuss some of this stuff at the F2F. Looks to be on the agenda for the 2nd day, as noted in w3c/webcomponents#713.

@tomalec

This comment has been minimized.

tomalec commented Mar 7, 2018

Tokyo F2F: The outcome was we will not move forward with this proposal. Overloading <template> tag with more functionality (like shadowroot attribute) may end up with lots of confusion for developers, especially given ongoing, not finalized progress with template instantiation. Specific new <shadowroot> tag would require new parser macro to wait for the end tag to attach shadow root and remove the node, that could introduce similar security problems during implementation as it did for template element.

We need to make sure that before shadow root is attached its <style>s should not apply to the outer/host tree, scripts and custom elements should not have an access to the <shadowroot> ascendant node.

I actually do not feel competent enough in the parser area to give a correct rationale why do we need that macro and wait for the end tag.
@rniwa could you comment shortly why can't we attach shadow root to the parent and remove the element on start tag, then continue attaching descendants to the shadow root, or beside using the different name, use the mechanics of template element for parsing?

Supporting non-scripting/scripting-forbidden environments is not sufficient motivation for implementing new features into the Platform for the Group. All other cases should be solvable by a library/custom element that implements declarative shadow dom. Therefore it's up to the user-land and frameworks to adopt such convention.

@prlbr

This comment has been minimized.

prlbr commented Mar 7, 2018

Supporting non-scripting/scripting-forbidden environments is not sufficient motivation for implementing new features into the Platform for the Group.

Was author-friendlyness considered? The consequence of the decision isn’t just that style isolation won’t work in non-scritping environments. It also means that style isolation is programmer territory only – it’s inaccessible to people who just use HTML and CSS, if I understand correctly.

@hayatoito

This comment has been minimized.

Collaborator

hayatoito commented Mar 7, 2018

Thanks for feedback. We are totally aware that Declarative Shadow DOM would address some use cases, such as an alternative of scoped css.
However, our consensus at Tokyo f2f is that we are not enthusiasm to update the parser only for supporting Declarative Shadow DOM. As @tomalec mentioned in #510 (comment), supporting non-scripting environment is not sufficient motivation for updating the parser, for a while.

@annevk

This comment has been minimized.

Member

annevk commented Mar 7, 2018

Closing this given implementers were opposed. This might make a comeback of sorts with declarative custom elements, see https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Declarative-Custom-Elements-Strawman.md for a sketch, and if we ever do something like "XML5" so we have a parser that we can tweak more easily than the HTML parser.

@annevk annevk closed this Mar 7, 2018

@justinfagnani

This comment has been minimized.

justinfagnani commented Mar 7, 2018

@prlbr yes, but there are enough tricky questions about how this should be behave that it seems like we should see more user-space solution uptake in the wild. There's not so much that the browser can do here that can't be done with a very small script and <template>. I think if someone vended a library for this and it got uptake it might have a better chance of a comeback.

The code should be very simple, something along the lines of:

for (const template of document.querySelectorAll('template[shadowroot]')) {
  try {
    const mode = template.getAttribute('shadowroot');
    const root = template.parentElement.attachShadow({mode});
    template.remove();
    root.appendChild(document.adoptNode(template.content);
  } catch (e) {
    console.error(e);
  }
}
@caridy

This comment has been minimized.

caridy commented Mar 7, 2018

I don't think we all realize the implications of this decision. Yes, there are many hacks that we can do, but each of them will have implications and limitations.

The main situation here is that without a proper mechanism to process a fully consumable HTML that can "eventually" be upgraded by registering the components will put WC in a very precarious position with respect to all other frameworks, and will be a deal breaker for many to not only avoid writing Web Components, but to avoid use/consuming Web Components, which is the most concerning part of it for me.

Here is my wish list:

  • I want it to produce HTML files that are indexable by search engines first and foremost.
  • I want my HTML to show consumable content immediately (scripts/modules are deferred because that's how we roll these days).
  • I want to encapsulate styles for the different parts of my HTML.
  • I want to upgrade parts of my HTML to enhance them so they become interactive.
  • I want my HTML to be resilience even when rolling with slow or flaky networks.
  • ... I also want a pony!

@justinfagnani's suggestion falls short on multiple items from that list, and I must say that his example is the most compelling solutions that I have seen so far, but unfortunately it's just not good enough.

I did not make it to the meeting, but I will like to get more details about the compromise around <template> with shadowroot attribute. It sounds like a good compromise, and users' confusion does not hold any water here, users are already confused with <template> in general.

@tabatkins

This comment has been minimized.

Contributor

tabatkins commented Mar 7, 2018

Yeah, this getting rejected is pretty bad. Shadow boundaries are a major part of making CSS more usable in the modern world, and making that declarative is a huge part of making this usable in the large.

That said, the declarative custom elements strawman does seem to fit my needs.

@caridy

This comment has been minimized.

caridy commented Mar 7, 2018

@tabatkins the declarative custom elements strawman definitely helps with the CSS but it is probably not going to help with some of the other use-cases. The primary difference between both of them is that a declarative custom element defines a new tag name, and upgrades all instances of it, while the declarative shadow dom simply allows declarative encapsulation without preventing an eventual definition and upgrade of the element.

@justinfagnani

This comment has been minimized.

justinfagnani commented Mar 7, 2018

@caridy something close to the snippet I posed would pretty faithfully the latest iterations of the proposal, so how would it solve your points any different than a native implementation? Aside from working with JS completely turned off, there's not much the browser can do differently from user-land code. Parser changes seemed to be completely off the table.

@caridy

This comment has been minimized.

caridy commented Mar 7, 2018

@justinfagnani no, it doesn't:

I want it to produce HTML files that are indexable by search engines first and foremost.

If it is not part of the native parsing, and everyone is doing whatever mechanism they choose to attach those shadow roots manually, how search engines can reliable determine that those templates should be analyzed and slotted? On the other hand, if is it native, spiders parsers can match the behavior of the browsers including the slotting algo.

I want my HTML to show consumable content immediately (scripts/modules are deferred because that's how we roll these days).

when is your snipped executed? at the beginning? at the end? per instance (sibling of the <template>)? it might be a good enough compromises for some, but not for all. There is a flickering effect if it is at the end, specially if you have styles that can affect the host, etc. So, it is not really solving this.

I want my HTML to be resilience even when rolling with slow or flaky networks.

Falls short for no JS, or slow networks, or any other similar scenario. In does case, the content will not be consumable.

@justinfagnani

This comment has been minimized.

justinfagnani commented Mar 7, 2018

The markup is the same as the <template shadowroot> versions of the proposals, so the parasability is literally exactly the same. We'd need a polyfill anyway. Standardization helps, but isn't a sure thing. For instance, do we know that crawlers properly don't consider template content now? Conventions, even if not in a W3C spec can help.

when is your snipped executed?

I think sophisticated enough JS can pretty reliably emulate what the browser would do. The browser can't stream contents into the shadow roots, so it'll have to be at least chunked, if not done in one shot if the whole page is in a shadow root. Flickering can be solved as the browser would, by processing in a bottom-up up order, so that all content is still in a template until the top-level template is hydrated.

Falls short for no JS, or slow networks, or any other similar scenario. In does case, the content will not be consumable.

If the snippet is inlined with the page, it'll be fine for slow networks. For no JS situations it won't render, but the content will be there for parsing. Renderers tend to have JS though. This is the situation we'd be in if this was a standard not implemented in every browser or crawler: we'd need a JS polyfill.

@tomalec

This comment has been minimized.

tomalec commented Mar 8, 2018

Flickering can be solved as the browser would, by processing in a bottom-up up order, so that all content is still in a template until the top-level template is hydrated.

The flickering I experienced with my prolyfill custom element is for the cases like:

<div>
  <template is="shadow-root">
    <link rel="stylesheet" href="make-div-green.css">
    <div>I'm green</div>
    <slot></slot>
  </template>
  <div>I'm black</div>
</div>

It flickers even if I have preloaded this stylesheet or loaded it in different shadow tree, as it still needs to load it separately here.

Then to avoid the flash of unstyled content, I need to:

  1. set display: none to the host,
  2. attach shadow,
  3. wait for all resources to be loaded, which is tricky due to lack of link.loaded promise ,
  4. remove display: none from the host.

Why it's still not enough:

  • there is now the flash of no content, as I cannot block on those links,
  • (needless to mention lack of stable environment or polyfill with customized build ins)
@keithjgrant

This comment has been minimized.

keithjgrant commented Mar 8, 2018

I am disappointed this has been rejected. I hope everyone realizes this is the sort of response that this gets from the web development community:

I am sooooo glad I never really bet on the platform. React continues to innovate, while the platform continues to stagnate.
https://twitter.com/AdamRackis/status/971758743011635202

Not that I agree with this sentiment, but… I understand where it comes from

@diervo

This comment has been minimized.

diervo commented Mar 8, 2018

As a non implementor, and someone that was in the meeting pushing for this proposal (and we do have strong use cases), I believe the push back from implementors was understandable at least with the current proposal (mostly the parsing concern and the idea that most of this could be done in user-land to a certain degree).

IMHO tweets like the one you mention don't precisely help in moving things in a good direction.

I think is up to us to give implementors more use cases and more ammunition to convince them that, this proposal, with whatever modifications, will have a indeed a huge benefit and its worth pursuing.

So we will continue pushing by giving strong reasoning, more use cases and variations on the proposal that will be easier to implement (if possible). At least thats how I believe we can move the platform forward.

@Hotell

This comment has been minimized.

Hotell commented Mar 8, 2018

Very nicely said @diervo ❤️ ! It's so low to just throw sh*t out on someone and flaming with tweets like that. very sad indeed.

@treshugart

This comment has been minimized.

treshugart commented Mar 9, 2018

@diervo, @tomalec, I'm interested in creating a single place for this where we can experiment on ideas, and possibly provide an official stance for an implementation. It'd also be good because those who want it know that the people who have participated in trying to push it will be operating in their best interests. This might be enough to help convince implementors over time.

Thoughts?

I am more than happy to use the skatejs org on GitHub (or even add it to the skatejs monorepo). Will invite anyone as maintainers who want to help.

@tomalec

This comment has been minimized.

tomalec commented Mar 9, 2018

Great idea! I would definitely love to share my ideas and experience with others in more dedicated place.

Dedicated repo would be beneficiary, so we could iterate on proposals, provide various prolyfills, track individual problems in separate issues, etc.

Staring it at Skate org could give some recognition from the start 👍, but I think it would be nice to make it less opinionated and more framework agnostic, to reduce hype bias. So framework A fans would not disregard it just because it's framework B thing.

I think about transforming my proposal repo to a new WICG repo, given the discussion already took place at https://discourse.wicg.io/t/declarative-shadow-dom/1904 That would also give us some IP, licensing security from the start.

I just don't know if that's a correct flow from W3C perspective, as implementers already expressed lack of interest in implementing this proposal.

@annevk

This comment has been minimized.

Member

annevk commented Mar 9, 2018

This is not the W3C, it's the WHATWG, but anyway, exploring things in a WICG repository seems reasonable. Having said that, I suspect you'll have the biggest chance of getting something to change here if you can demonstrate widespread usage of a polyfill or equivalent workaround.

@treshugart

This comment has been minimized.

treshugart commented Mar 9, 2018

Could also use the webcomponents org; probably better than the skatejs org anyways. I can create a repo there if we think that's a better spot.

@annevk that's the end goal. My reasoning for proposing we put it somewhere "more official" is simply so that we can underscore the community support for it, both for people wanting to use it and, ultimately, for bringing it back to the table.

@Jxck

This comment has been minimized.

Jxck commented Mar 12, 2018

I wanna see the comment for below by @rniwa about what will happen and concern to parser for understanding problem correctly.

@rniwa could you comment shortly why can't we attach shadow root to the parent and remove the element on start tag, then continue attaching descendants to the shadow root, or beside using the different name, use the mechanics of template element for parsing?

#510 (comment)

@rniwa

This comment has been minimized.

rniwa commented Mar 12, 2018

We added templete element support to WebKit back in 2011 (even before the Blink fork by a Google enginner). Even as recently as a couple months ago, I was fixing security bugs related to template element in WebKit due to the way it moves the child nodes into an inert document.

The proposed semantics (of attaching a shadow root after parsing the start tag) is equally problematic because now the HTML tree builder algorithm has to be aware of shadow trees, and has to take it into account when looking at the list of open elements, etc... Because ShadowTree and Document are special root nodes of a DOM tree in WebKit (and Blink) which are directly referenced by every node in the tree, this would mean that we'd have to inspect and fix every HTML parser code to correctly update the root node (Document or ShadowRoot) referenced by a newly parsed node.

Similarly, we recently removed isindex tag which was a parser macro that expanded into a form element and an input element. This feature introduced numerous security bugs in our HTML parser due to the fact isindex itself doesn't generate an element, and removing the feature was considered a big security win. The proposed semantics of shadow element has this problematic behavior (since shadow isn't a real element).

@tomalec tomalec referenced this issue Oct 25, 2018

Open

HTML Modules #645

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment