Alternative proposal for closed shadow DOM

[mrmr1993]

The closed flag for web components doesn't seem to have a clear purpose, and has several issues.

Security:

• Current security of closed components from the page is illusory
  • A page can hijack the registration of closed components (via Closed flag proposal breaks ability to audit and automate tests of web pages #354 (comment)) with

var originalAttachShadow = Element.prototype.attachShadow;
Element.prototype.attachShadow = function(options) {
    options['mode'] = 'open';
    return originalAttachShadow(options);
}

• They share a javascript context with the main page, so there's a reasonable chance of accidentally leaking the shadow DOM.
  • hooking Element.prototype.addEventListener is an easy example, and can't be closed without some webcompat breakage.
  • The Proxy object provides similar opportunities when hooking getters.
• Security is asymmetric.
  • The host page has to implicitly trust the component with its important data (session cookies, password fields, etc.).
  • The host page cannot easily place any restrictions on the behaviour of the scripts governing the component.

Integrity:

• As above, the shared javascript context creates ample opportunity for leaking the shadow DOM.
• The host page has no guarantees that the web component code will not interfere with the it outside of the component element.

Automation, accessibility & testing:

• Currently spec breaks all of these. See Closed flag proposal breaks ability to audit and automate tests of web pages #354.

Taking these into account, it seems that a closed shadow DOM is little better than an open shadow DOM in any of the aspects it might hope to be, and causes problems for outside interaction with pages.

Proposed alternative:

• Keep majority of shadow DOM spec.
• Remove ShadowRootMode and all of its consequences from the shadow DOM spec.
• Introduce a new element (say <component-iframe>) with the same attributes, DOM interfaces and behaviour as <iframe>, except
  • children of <component-iframe> are assigned to <slot>s in the document tree of the <iframe> using analogues of the slotting algorithm (§3.2), assigned nodes algorithm (§3.3), and distributed nodes algorithm (§3.4).
  • a DOM interface is added to the <component-iframe>'s window for registering API functions for the component, e.g.
    • window.registerComponentMethod(name, apiFunction) to add a method
    • window.unregisterComponentMethod(name, apiFunction) to remove a previously added method (functionally similar to removeEventListener elsewhere).
  • a DOM interface is added to the <component-iframe> HTMLElement for accessing the registered API, e.g.
    • HTMLComponentIframeElement.getRegisteredComponentMethods() to list the added methods
    • HTMLComponentIframeElement.NAME(...) where NAME is the name argument for a method added via registerComponentMethod.
  • the <component-iframe> should begin loading before being inserted into the document
    • this allows it to process API calls, perform initialisation, etc. before it is displayed to the user.
    • this could be achieved by adding a load method to <component-iframe> which creates the nested browsing context (which persists after insertion into/removal from a document) and processes the iframe attributes (see iframe spec).
    • this allows the component respond to API calls and perform initialisation before it is inserted into the document, if necessary.

Is this reasonable or even doable as an alternative?

/cc @dylanb, you're the major voice against closed shadow DOMs in #354. Would this proposal address your concerns/is it something you could get behind?

[annevk]

The purpose of closed is to avoid accidental dependencies on the internals of shadow trees. We do want truly isolated shadow trees at some point, but as you indicate those require a different design. They're not considered an alternative to closed shadow trees however.

[mrmr1993]

The purpose of closed is to avoid accidental dependencies on the internals of shadow trees.

Doesn't grepping for shadowRoot do this without breaking our accessibility, automation and testing tooling?

[annevk]

None of accessibility, automation, or testing tooling should be broken as already discussed.

[mrmr1993]

.. as already discussed.

Doesn't seem to be much discussion of this or any planned fixes on the Chrome side. Could you link me if I've missed it?

Revisiting:

The purpose of closed is to avoid accidental dependencies on the internals of shadow trees.

Are we really talking about carving off parts of a web developer's DOM that they are responsible for, aren't allowed to access, because they might make a mistake? Not to mention that a developer might want to create intentional dependencies on the internals of shadow trees, regardless of whether the component author intended for them to.

I really can't see who's benefiting from this.

[hayatoito]

Regarding security, Shadow DOM is not a security mechanism. That has never been a goal of Shadow DOM, either open or closed. Do not use Shadow DOM if you want a true security.

Should we mention that somewhere if people are likely to misunderstand Shadow DOM's role?

[rniwa]

Doesn't seem to be much discussion of this or any planned fixes on the Chrome side. Could you link me if I've missed it?

Such discussion has to happen at each browser vendor and/or for the WebDriver specification, not here.

Not to mention that a developer might want to create intentional dependencies on the internals of shadow trees, regardless of whether the component author intended for them to.

This is precisely why closed shadow tree is needed. Because of this possibility, once a component is published and used by third parties, there is no way for component authors to know whether there is any use of a given component that depends on a particular shape of its shadow tree. Closed shadow mode makes it much less likely for any use of the component to depend on its shadow tree structure.

[dylanb]

Is all of the effort to try to patch all of the supposed "leaks" of information about the closed shadow DOM, plus all of the other changes to specs and implementations (e.g. webdriver) really worth the questionable benefits of the writer of some component, trying to stop the users of that component from being idiots?

My experience shows that bad coders will be bad coders, regardless of how much you try to stop them from doing it, and in this case, the only people that get "hurt" are the bad coders. So why all the effort?

[annevk]

I think it's rather disrespectful to name those people "idiots". They might simply have different priorities. Guarding against typical mistakes is useful and something browsers deploy all the time. Making it available to others is useful.

[rniwa]

First off, terms such as idiots shouldn't be used lightly per our Code of Ethics and Professional Conduct. The term is discriminatory towards people with intellectual disabilities. See campaigns like http://www.r-word.org.

Now, encapsulation is the pinnacle of modern day programming. The problem here isn't so much that some developers would poke into shadow trees of various components accidentally or purposefully. It's that component authors can't have any reasonable expectation that users of their components wouldn't do so.

Why is this bad? Because those components would need to be updated in the future. And when they do get updated, all those uses of older versions of components that poked into their shadow trees would break.

Look no further than shadow piercing combinators, which was introduced as a convenience mechanism for developers to pierce through shadow boundaries to style elements. At the end, Blink removed this feature from the engine because authors abused it.

Look no further than the fact we can't even add Node.prototype.rootNode or Event.prototype.deepPath due to Web compatibility. This WILL happen to your components once enough people start using them.

[dylanb]

@rniwa, seeing as those comments were directed at a theoretical person and not at anyone on this list, or any identifiable group of people, I cannot see how that violates the code of conduct

[dylanb]

@rniwa the fact that authors used the shadow piercing combinators should have been a clue to the blink developers that maybe there is a need here that is not being met. Removing a feature, because it gets used "too much" seems to me to be the exact antithesis of agile, UX-based development.

This is precisely the problem with this part of the spec, and more broadly with the HTML spec. It is too dominated by the browser vendors opinions and not focussed on the needs of the users.

[BronislavKlucka]

Hello

first of all, as a starting point, we should consider 3rd party
components basically the same way as user-agent provided components, as
a black box. There is no need for different approach, that would be even
counterproductive. What you are describing can be applied to standard
elements as well. So what, we live with that for 1/4 of a century.

security

never been an issue, because that is not what ShadowDOM (SD) is for

• rewriting attachShadow is not problem with SD, it's ECMAScript (ES)
  feature, in such case we should not be able to write any ES, because
  every method can be hijacked.
• yes, host page have to implicitly trust component with data, but as
  a desktop developer, every time regular desktop developer uses 3rd party
  component, the security risks are high (e.g. direct access to harddrive
  on regular app). But one should be aware of what one is using. We do not
  take down internet because someone might download virus, it is everyone
  responsibility. Such is this. We should not limit valid usage because
  author might misuse it and programmer, who should be even more security
  savvy than regular user, does not care

Integrity

• component element might have very reasonable need to interfere with
  outside of itself through public API of outside, the same way outside
  can interfere with the element through its public API or even other way
  (e.g. putting stylesheet link element, script element will interfere
  with outside, we have those exactly for this purpose)

automation, accessibility & testing

I see no problem. You can test 3rd party the same way as you test
regular elements. It might be poorly written (insufficient public API,
not enough events, ...] , yes, but that is not problem of closed SD, but
that particular component. The same applies for the rest of this point

desktop component development / usage / behavior

Desktop development (and not just desktop, but I'm thinking controls
mostly) is used to this concept for a very long time. ES is bit
different, but other languages know concepts of private or protected
parts of objects (I'm missing those in ES). Which actually turns
everything into black boxes. Every problem with this is of 2 kinds

• author creates poor public API, developer cannot use the part of
  object that should be usable from outside
• developer is simply using wrong object (the object simply does not
  fulfill all needs)
  neither is the problem with the concept of privates/protecteds (= closed
  mode for web dev), but, as usual, with people.
  This is not about "but this is web page, not desktop", first of all,
  every one of us have been doing it for as long as we are 'web
  developing' with user-agent provided elements and second of all, concept
  of 'everything touches everything, whenever and however it wants' may be
  great for web presentations, but is detrimental for web apps, because
  prevents reusability, composition and update of components. All those
  concepts rely on blackboxing of components: author will provide you with
  enough API to use the component and makes sure it will not change (or
  the behavior) with upgrades, and you will not rely on internals so it
  can actually be updated/improved. Save composition relying on either
  being design to interfere by author (which developer should know) or on
  explicit command from developer. I believe we have enough issues through
  the years with 2 frameworks on one web page interfering.
  The hardest possible black-boxing of components is simply necessary for
  creating truly reusable components, may be not natural for web
  developers used to total control over everything, but me, developing in
  different environments as well, I was actually missing this = easy
  reusability of components across several apps. (can anyone add privates
  and protecteds into ES? At least we have WeakMaps and symbols, but
  protected properties/methods would make my day... I know, TC39 issue)
  • just a notice, controls in desktop environment (unless some composed
    frame) are all actually canvases with the states of the control painted
    onto it (simply thing of checkbox), there are no internals one might go
    nuts with.

Brona

[mrmr1993]

The shadow DOM is an amazing idea, allowing developers to package up a 'web component' in a self-contained way, so that it won't unintentionally be affected by what's outside. This is great for everyone: the component developer can develop in the knowledge that their component won't be unintentionally affected by the page, the page developer can use the component in the knowledge that it will work correctly if they use it as intended, and the component can be updated safely without breaking anything. I think we are all in agreement on this.

We seem only to hit a problem when the page developer (or a third party interacting with the page) wants to intentionally change the DOM within the web component. Then, we have 2 choices:

• The web component is still separate, but may be accessed by code not belonging to the component developer, via shadowRoot. Currently this is an "open" shadow DOM.

  The web developer can always realise their intentions by breaking into the shadow DOM if they need to, but do so at their peril, since things make break immediately or at some point in the future. Again, I think we are all in agreement that such a thing should exist.

• The web component is completely separate, and cannot be accessed by any code not belonging to the component developer. Currently this is a "closed" shadow DOM.

  This is being justified by asserting the importance that the web component always works correctly, everywhere. Implicitly, we are saying that it is more important that the web component works correctly in a developers page than the developer being able to do what they are trying to do.

Let me repeat, we are saying that it is more important that the web component works correctly in a developer's page than the developer being able to do what they are trying to do. This is blatantly absurd.

Add to this the fact that, in order to protect the developer from themselves, we have to create an exemption in every third-party interface that should have access. (This must be nearly all of them, since it is hard to reason about a page when chunks of it are locked away from view behind myriad opaque unstandardised interfaces.) And then the page developer can easily disable closed shadow DOMs anyway, even though they are the only ones we want to lock out. This takes a bad idea and inflates it into a multi-specification disaster.

I appreciate that a lot of work has gone into closed shadow DOMs, but can we please now reconsider, and begin taking steps to unwind it.

[andyearnshaw]

Add to this the fact that, in order to protect the developer from themselves, we have to create an exemption in every third-party interface that should have access.

Nobody is forcing developers to close shadow trees. It's entirely their choice. If it causes them trouble, they can easily open them in the source code and forget all about closed shadow trees. Alternatively, if they want to access closed shadow trees during testing, they can hook attachShadow quite easily as you pointed out in your opening post. Likewise, any developers on the web can do this to explicitly gain access to a component and modify it.

I, for one, am very keen on closed shadow trees. They're a safety mechanism on a footgun and they can provide comfort to developers that they can safely make changes to their components. Taking the safety off should be as inconvenient as it is, as developers who take the time to learn how to do it are likely to learn the risks in the process.

If you don't like the closed flag, simply don't use it.

[BronislavKlucka]

Matthaw,

I do not want this to become our own discussion, but I'll try to respond
one more time, hoping to make this point, ultimately it's up to others
to decide

1/again, we have never been able to do what we wanted on the page,
internals of user-agent own elements have always been hidden from us, no
difference here.
2/ having object being just a container for methods and properties has
always been a bad idea, most OOP languages have at least privates to
hide internal states (it makes no difference whether it is field or DOM)
and for decades we've tried to do this in ES. Aren't you annoyed every
time you have to consider whether to attach everything on object and use
constructor as a closed box around internals (but risking huge memory
usage) or put things on prototype, but then leaking everything to
outside word? I do not know your experience, and it's not entirely
correct argument, but my experience of 25 years with sealed languages
and 20 years with mishmashed web pages left me with deep feelings, that
this concept of 'everything sees everything' is deeply flawed (I miss
scoped styles...). This is simply conceptually wrong.
I wander... have you protested on WeakMaps or Symbols at TC39? Those
allow to create privates for plain ES... Where is any difference?
3/ Yes! It is more important, that the component's working on authors'
page, than that developer wants to do whatever they want to do (and by
hooking attachShadow or even altering source code they still may). It's
his/hers component :). This simply creates a contract, contract between
a author and all developers using this component (not just the one who
want to screw it), it creates contact among different components on the
page, what they can or cannot do to each other. This is, how it should
have been from the beginning and we should be here discussing whether we
need open mode... not the other way around. "Open" is, what's weird
about ShadowDOM, not "closed".
But I can accept both, because I will assume, that by using open
component, I can touch inners, author is most likely sure, it will not
cause any problem. But when I will see closed one, I will assume, that
it is simply danger.
4/ And again, if you ever find yourself relying on the internal DOM of
Shadow, either it's poorly written or you are using wrong component.

This is simply both theoretically and practically correct way.

B.

[mrmr1993]

we have never been able to do what we wanted on the page,
internals of user-agent own elements have always been hidden from us, no
difference here.

@BronislavKlucka As I mentioned above, the problem is that the resulting API will be unstandardised. If the component is incomplete/doesn't satisfy your requirements, you have to modify the code for it (if the licence allows you to) or write your own from scratch. With user-agent elements, the implementations are generally pretty complete (notable exceptions include selection API for some newer <input> elements, etc.).

For example, if a closed shadow DOM forgets to register an API for addEventListener("input", ...) and your application depends on it, I think it's a bad thing to try and stop you from working around it, even if it may cause breakage later.

I wander... have you protested on WeakMaps or Symbols at TC39? Those
allow to create privates for plain ES... Where is any difference?

This is the DOM, not application logic. What we're talking about here is carving off parts of the user interface that the developer isn't allowed to affect. You're being made to hand over end-to-end control of user interaction and experience with parts of your application. In my opinion, this is worlds away from hiding members of behind-the-scenes libraries in an abstract object, and is not something to be desired.

This simply creates a contract, contract between
a author and all developers using this component (not just the one who
want to screw it), it creates contact among different components on the
page, what they can or cannot do to each other.

Why must it be the developers of the component that get to enforce the contract? The component exists in a vacuum without the applications that include them, and keeping or breaking the contract only breaks that application. It seems clear to me that the application developer should be the one who has control over how stable/unstable, broken/unbroken and functionality-rich/-poor their application is.

The component is not more important than the application; it should be subservient to it. We seem to be rushing to forget that.

if you ever find yourself relying on the internal DOM of
Shadow, either it's poorly written or you are using wrong component.

I completely agree. Problem is, "closed" looks like a big ribbon to wrap around your component when you decide that it's done.

You can quite happily write a component once, to a spec that fulfils your needs, and consider your project complete. You think, because you've matched your spec, that nobody will need to get inside it. Anyway, they might break it, and you might look bad if they do. So you tie it up with the nice "closed" ribbon and send it out into the world. It might be great at what it does, end up well-tested from widespread use, and work as intended. But you've made the decision for developers using your component that it is complete, regardless of whether it does what they're actually trying to use it for. This is not the decision you should have made, and you've stripped developers of opportunities for no good reason.

A more complex point is bundling. Say you've created the best spreadsheet web component ever, and you'd like to heavily brand it, or to advertise within it. By making it a "closed" web component, you're telling the application developer that they have no choice in this -- that they must allow you to show their users these in your application, regardless of whether they disagree. I want it to be that the application developer retains final creative control, whether the web component developer likes their decisions or not.

[andyearnshaw]

But you've made the decision for developers using your component that it is complete, regardless of whether it does what they're actually trying to use it for.

As the developer of a component, the writer and copyright holder of a piece of software, isn't that my prerogative? You're telling me that I shouldn't be allowed to decide how my component is used?

A more complex point is bundling. Say you've created the best spreadsheet web component ever, and you'd like to heavily brand it, or to advertise within it. By making it a "closed" web component, you're telling the application developer that they have no choice in this -- that they must allow you to show their users these in your application.

You don't need a closed shadow tree for this, a restrictive license would be enough. Look at the FlowPlayer license, for example, and FlowPlayer's component doesn't use shadow DOM (yet).

If a component is truly open source and its shadow tree is closed—and this bothers you—, fork it and do whatever you want with it. If the component is not truly open source and its shadow is closed, then you can either get around the closed tree the "hard" way and modify it while staying in the terms of the license or move on and don't use it.

[dylanb]

Openness is the reason the Web is as successful as it is. If the inventors of HTML and JavaScript had decided to create closed systems, we would not have half of the features we currently use every day. We would not have any of the HTML5 shims that allow us to use cool new features while still supporting older browsers for example.

There are thousand of innovations we all use every day built on top of the technology that get this technology to do things that the inventors never were able to predict.

ReactJS is an amazing invention. I don't think I could have invented it. The React object uses prototype to expose its functionality to users. If it were not possible to override the prototype of a third party component, I would not have been able to create this module https://github.com/dylanb/react-axe and this module https://github.com/garbles/why-did-you-update which uses the exact same technique I used, would not have been able to have been created either.

The ReactJS developers are awesome! But they could not have predicted the existence of those two (and probably hundreds of other modules) that provide value they did not see.

I have never once wanted more obfuscation. I have only ever wanted more transparency and more ways to look into and manipulate things other people created. For example, right now, I wish that there were a way to find all the event handlers registered on a page (without having to write an extension) because this would allow me to extend the axe-core accessibility evaluator to do things it currently cannot do in a cross browser way.

I wish that this team would expose the composed tree in a way that it can be inspected using the DOM API. I wish that the accessibility mapping information of all nodes were exposed to JavaScript so that we could write automated cross-browser tests to see whether there are accessibility bugs in UA implementations. I want more openness every day, so I can do things the browser vendors have not thought of and in many cases are not interested in doing themselves.

@annevk you can repeat your assertion over and over that your are not breaking the accessibility testing but that is not what @mrmr1993 has so far found as currently being discussed here #354 (comment)

@andyearnshaw It is not worth sacrificing the future innovation around Web Components just so you can prevent someone using your stuff in a way you could not foresee.

I agree with @mrmr1993 that it is the inadvertent use of this flag that is most dangerous. As shown by some of the comments above, developers often think they know it all and they know better and they want to control the whole world. This desire, as shown by all the great innovations that have happened because of the openness of the Web platform, is misguided.

Firefox trounced IE because it was open and Chrome would never have succeeded in winning over the development community if it had been a totally closed system like old IE. Microsoft has recognized their folly and as a result, Edge is heading in a more open direction. Open systems win when there is free competition - always. Web Components will compete with other mechanisms for achieving the same things (like React for example) and if it is not open, it will lose.

[annevk]

@dylanb that implementations are imperfect does not mean the feature is broken. False equivalence.

[andyearnshaw]

The React object uses prototype to expose its functionality to users. If it were not possible to override the prototype of a third party component, I would not have been able to create this module ...

ReactJS is open source.

@andyearnshaw It is not worth sacrificing the future innovation around Web Components just so you can prevent someone using your stuff in a way you could not foresee.

I think that's more than a little extreme, to be honest. Plenty of JavaScript and HTML components have not been open, for instance those that are heavily obfuscated or use cross-origin iframes. They haven't hindered anything.

While I stand by what I said about authors rights to restrict use of their software, I'm more interested in that extra barrier that means people are more likely to know the risks before modifying the component, and I could issue structural changes to fix minor issues without worrying about breaking existing live instances. If someone can easily write foo.shadowRoot.insertBefore(bar, foo.shadowRoot), then a small, minor issue-fixing, structural adjustment to the component requires a major version bump to our library to represent a backwards-incompatible change. Our code is open to our library users, however, so there's nothing stopping any of them from taking the source code for a component and repurposing it to fit their needs.

[BronislavKlucka]

@mrmr1993
you're still dealing what you want, not what is correct approach from OOP point of view. As already mentioned, closed flag does not prevent you to do whatever you want with that component (licence does), you can change it to open in source, you can hijack attachShadow. It's not there to prevent you to touch it. It simply say something about the internals. I believe @andyearnshaw explained the difference between flag and licence quite well.

Using internals for plain object and for ShadowDOM is exactly the same. There is no difference, it is private logic, you can expose private property in object as well and face unintentional consequences, because the object never assumed that this particular property can be changed outside of internal logic. The fact that end user does not see object, but sees a control makes no difference.

Let me give you example of difference between open and closed based on usage, not on what anyone wants.

Let's say I have application with calendar and list of events in selected day.
The calendar is clearly a control (closed), you should not be able to touch it, delete a week, move Tuesday after Wednesday... And that is what the closed says. And then there are 2 scenarios. I'm either a poor developer or my intentions are only to use it in this app for Czech users, so I will hard code Czech day names, week starting at Monday etc. Which would make it most likely useless for you. If I were to release it under some OS licence, you can fork it regardless of the flag, If I were to release it under closed licence, you cannot do anything, regardless of the flag. Or I intend it to be full blown reusable control, so I implement internationalization, I implement some sort cellDisplay callback with some object as a parameter that will allow you to control font, borders, background, I can pass DocumentFragment so you can create overlays. This is up to possibility of the control, but the closed flag says, that the internal DOM structure should not be changed, that I may rely on consistent logic of that structure, the same way as private properties, methods of ES code tell.

The event list, the list, that is displayed when day is selected is different story. It's a box with date/time of the event, name, location, description, maybe with some control buttons... It's something that in desktop programming is called frame (poor name for Web dev. though :(, component module? object view? control box? ). It has some properties of control (object). It Is instantiated more than once, in all cases it serves the same purpose but with different attributes (display of event, but different events), it has properties of its own (e.g. background) method of its own (displayEvent(Event event)), it has its internal logic (if I click delete button, event gets deleted, no need to code it outside), but there is no need do shield it. There is no need to protect internals, programming that I should not expect the DOM to exist as designed at any time (always test for existence of element, in closed, good practice, in open mandate). If "the outside" will want to change internal structure, why not... (in desktop programming the frames' controls are by default public/published). It's just a view.

I hope you can see the difference based on usage / OOP theory, not whether we like some programming paradigm or not.

And yes, opinion of the autor is more important, that opinion of the developer/user of the component, whether it's expressed by licence (which you cannot touch) or closed flag, which you can touch (based on licence)

@dylanb
You are confusing openness of the technology, with openness of the licence. You cannot copy some web page without authors' permission regardless of the fact, that it was written in HTML/CSS/JS (open technologies). Your post is more like plea for OS licences than open technologies. As mentioned several time, the difference is about licence, not the flag. The closed flag is not about "you never ever can because I, author, am the smartest one", the same way declaring property as private is not. It simply says "here you have public API, here are my privates" you should not touch that. But if released as OS, do what ever you want in the end....

[dylanb]

The React object uses prototype to expose its functionality to users. If it were not possible to override the prototype of a third party component, I would not have been able to create this module ...

ReactJS is open source.

How useful would those two modules be together if you had to modify React source code to add those features, keep syncing this with the main fork and keep updating it every time you want a change from another module or want to add a new module etc. Talk about upgrade difficulties.

If someone can easily write foo.shadowRoot.insertBefore(bar, foo.shadowRoot), then a small, minor issue-fixing, structural adjustment to the component requires a major version bump to our library to represent a backwards-incompatible change. Our code is open to our library users, however, so there's nothing stopping any of them from taking the source code for a component and repurposing it to fit their needs.

Why are you so worried about backwards compatibility for people who do this? If you do foo.shadowRoot and your code reviewers allow you to merge it, you must have a good reason and you should put in place tests and release strategies to make sure it works every time you ship. If you don't, then you have no-one else but yourself to blame for it. If you have people on your team committing this sort of code and you don't like it, you don't fix that by going to the standards body and asking them to disallow this. You fix it by changing your coding standards and the culture of your team - or voting with your feet.

[dylanb]

@BronislavKlucka I am not advocating for OS licenses. What I have said through illustration with many examples is that the ability to change and see and modify all aspects of the Web page (within the limits of security) is something that has allowed great and valuable innovations to occur and when we create features that close things off, we are limiting the creative flexibility of the users of those components. This will limit innovation and lead to possibly brilliant inventions not being technically feasible at all.

One aspect of your calendar that I might legitimately want to reach in and fix is to add missing accessibility information. If it is a closed source component that has a closed flag and I cannot do that, then my only options are to switch your component out, write an entirely new component or do without the calendar picker feature.

All of those alternatives are very expensive and could be avoided by simply not implementing a closed flag.

[BronislavKlucka]

@dylanb

All of those alternatives are very expensive and could be avoided by simply not implementing a closed flag.

no... I can write it in open mode with closed source licence... And if I choose closed source licence and my component will not meet your need, you still have to move on.. Or do you want to force on me, the author, the licence as well?

.... the ability to change and see and modify all aspects of the Web page .... is something that has allowed great and valuable innovations to occur...

and has been a plague over the years when trying 3rd party parts somehow not to leek out and the outside not to leak in

and when we create features that close things off, we are limiting the creative flexibility of the users of those components

which should be solely in control of the author, by licence

can we get back to problem at hands and not make it about author vs. developer, OS vs. closed licencing?
We got to the point, where the only argument against closed is based on "I want to do something, author did not want me to do". Which is mostly not argument at all. It has its real impact in e.g. your accessibility calendar issue, but that is problem of the calendar, not the problem of closed flag and the theory/reasoning behind it. I can implement whatever poorly and you will have to fix it, and do it with every upgrade, it does not mater whether it is ShadowDOM, piece of ES, library in Pascal, framework in C#... I can make SD opened and screw up ES implementation of component

[andyearnshaw]

If you have people on your team committing this sort of code and you don't like it, you don't fix that by going to the standards body and asking them to disallow this. You fix it by changing your coding standards and the culture of your team - or voting with your feet.

You misunderstand. The users of the library aren't part of my team, nor under my control. Nor do I, or any of my immediate colleagues oversee the work they do. In many cases they're external clients and have only a basic knowledge of web technologies. These are the ones I'd like to protect from themselves. Considering that the API is used in ads and a single client could have many (many) creatives with copy/pasted code, I think it's a valid concern. Frequent backwards incompatible releases would adversely impact our support channels and require us to maintain several versions of the product simultaneously.

If it is a closed source component that has a closed flag and I cannot do that, then my only options are to switch your component out, write an entirely new component or do without the calendar picker feature.

For custom elements using shadowRoot, you could easily subclass and prevent the super constructor from attaching a closed shadow root:

class YourElement extends MyElement {
    attachShadow(o) {
        o.mode = 'open';
        Element.prototype.attachShadow.call(this, o);
    }
}
customElements.define('my-element', YourElement);

[domenic]

I don't really want to get in between you two, but I want to point out that at least as of yesterday (haven't read through the latest replies) there was a big fallacy in this thread, which was that closed prevents the application developer from modifying the shadow tree.

This is never true. There is no DRM for JavaScript. Any JavaScript code that you deploy in your webapp, you have complete control over. You can modify it as you see fit, for example, by changing all occurrences of the string "closed" to "open".

Yes, this can be inconvenient. Yes, it means that certain libraries will exist which you might not be able to use out of the box without changing their source code first. That's fine: there are a lot of things a library developer can do to make their library less than usable, such as using an illiberal license, or naming their public APIs with gibberish, or obfuscating their source code. Such libraries will get less uptake by people who care about such things. That's OK.

The application developer is never locked out of control of their webapp by the existence of features in the browser, like WeakMaps or closed shadow DOM, that give encapsulation. The application developer can always change the code that uses those features to simply not use them.

[dylanb]

so let me understand the argument:

1. We need closed shadow roots to make it difficult for users to get around when we think they are going to hurt themselves
2. If you really want to get around it, its easy, just implement 7 lines of code,
3. Its a lot of work to implement closed shadow roots
4. This will have an impact on other specs like WebDriver
5. This is a worthwhile effort

[dylanb]

@mrmr1993 After thinking about your proposal: As long as it supported the mechanisms through which code can be injected into the iframe from a privileged context (as used by WebDriver), it would solve all the problems I currently have with the closed flag.

It would also have to solve some of the styling problems that iframes have in order to be useful though.

[BronislavKlucka]

@dylanb
no, you understand it wrong, your assumption is simply a false at the first point.; It is not about making author the greatest one and developer the incompetent one who can hurt him/her self. It is about saying whether the component relies on internal consistency of DOM or not.

[mrmr1993]

Following a slight tangent: is there any indication that those naughty developers making widespread use of shadowRoot will suddenly be persuaded not to because they'll need an extra 4 line hack?

[dylanb]

@mrmr1993 to extend that line of thinking...is it really worth all the effort just to change a property access into those 4-6 lines (depending on how you're counting) plus a property access?

[treshugart]

I'm torn about this. I 1000% can sympathise with both sides. I work on the standard UI library for my company and have seen some pretty awful things being done to the "shadow DOM" of our components that having a closed shadow root could have helped with. That said, many of these problems were done by accident - or because the precedent we as developers have set in the industry is that this is okay up to now - and having access to the shadowRoot would make expectations consistent for both first parties and integrations.

Maybe the pragmatic solution here is to have open be the default (also not requiring a user specify { mode: 'open') and continue the discussion about how to provide closed shadow roots for v2? After native shadow DOM is used in anger I think things will start popping up that could never have been predicted. I see no reason this should block v1 from shipping.

[annevk]

That would go back on implementer consensus. That was the position until last year and it basically meant that only Chrome would ever ship something resembling shadow DOM.

[mrmr1993]

That would go back on implementer consensus.

So this is non-negotiable?

That was the position until last year and it basically meant that only Chrome would ever ship something resembling shadow DOM.

To clarify, are you saying that open shadow DOM -- which is widely acknowledged to be advantageous -- will not be implemented without closed shadow DOM, which variously has:

• advantages (summarising from above)
  • a 'stronger contract' than open
    • in reality, this means that 5-ish more lines of code are needed
    • the people that the workaround makes uncomfortable (overriding a DOM API in production) are the people most likely to already understand the risks of penetrating a shadow DOM and have accepted them
    • copy-paste-ers will notice only that they had to do one more search to make their code work
  • (can't find any others)
• disadvantages
  • workarounds for 'code injectors' (variously extensions, user scripts, automation/testing tools, accessibility tools and bookmarklets) will either not exist -- generally accepted as unintended/bad -- or will result in
    • proliferation of identically intentioned APIs across multiple specs (which all inherit the DOM API anyway)
    • inconsistency across APIs, since each spec has to design and incorperate its own
  • asymmetry of implicit understanding
    • component developers can easily make an uninformed choice of 'closed', causing problems for downstream developers (as opposed to eg. Object.defineProperty to shield shadowRoot)
    • application developers have to patch a DOM API to undo this
  • goes against what is actually being done on the web
    • introduction of shadow DOMs have seen widespread use of shadowRoot
    • there is noticably little developer interest in closed shadow DOM in the wider community
  • lots of work is being done to implement a feature which will be
    • mostly unnoticed by developers who it wasn't intended to affect, and indistinguishable for them from open shadow DOM
    • irritating for the few developers who can't Google around it
    • completely circumvented (code completely unused) when developers work around it.

Maybe this is already written in stone and I'm just pissing in the wind, but I really can't see why there is such strong intention to have this 'feature'.

[annevk]

I think it's non-negotiable, yes. I've iterated that various times throughout these issues. I disagree with your characterization of the advantages and disadvantages. The web has encapsulation, see, e.g., closures. It's not a lot of work to implement this feature. Providing APIs to bypass it to privileged APIs is fine, as we already stated. It's not accepted as unintended bad. And not allowing downstream to override things is a feature. Advantages have already been listed earlier in this thread.

[mrmr1993]

I think it's non-negotiable, yes.

Is there a procedure for opening a negotiation in such a case?

Advantages have already been listed earlier in this thread.

With the availability of the workaround, the only difference with open shadow DOM is the method of access, no? AFAICT, that was the only advantage that factors in this consideration.

If there were others that do factor this in and that I missed, could you briefly summarise them?

[annevk]

Is there a procedure for opening a negotiation in such a case?

I guess you'd have to convince the implementers.

With the availability of the workaround, the only difference with open shadow DOM is the method of access, no?

I'm not sure what this means.

As for arguments, I would consider #499 (comment) and #499 (comment) to be pretty clear.

[dylanb]

@annevk so what you are saying, is that the implementors' input trumps the input of users?

As pointed out #354 (comment) the consensus you talk about was never documented. In other words, you did not follow W3C process and give others an opportunity to voice their concerns.

I also notice that none of you have commented on my summary of the arguments #499 (comment), I assume because it is pointing out how weak the pro-position actually is. The benefits of this proposal are minuscule, yet their impact in terms of things that break and implementation cost is huge.

[annevk]

@dylanb in practice, the buck ends with implementers. However, according to the W3C it's the Director, so I suppose you could appeal to them, but I don't think any of that will change much here.

I didn't really reply to your summary of the arguments since I didn't quite know what to make of it.

Anyway, I will unsubscribe from this issue now since I feel like I've been repeating myself too many times.

[rniwa]

The consensus to have no default was reached in a F2F meeting on April 24th, 2015:
https://lists.w3.org/Archives/Public/public-webapps/2015AprJun/att-0307/24-minutes.html#resolution08

This was one of the most controversial and contentious bit during shadow DOM API discussion:
https://www.w3.org/wiki/Webapps/WebComponentsApril2015Meeting

As you can see yourself, this is a very complicated topic of which people have varying opinions. This is precisely why we added a flag to toggle the two modes and didn't force the default mode on anyone by requiring each author to specify so that we may pick the appropriate default mode once we learn how this feature gets adopted by developers.

So on the contrary to what you're claiming, this is precisely the decision made to wait & honor the future feedback we get from developers (i.e. users of this API).

[treshugart]

@rniwa that is a very sensible approach indeed. I apologise if my comment was a bit naive.

Coming from a different angle, and arguing against the automated testing angle, wouldn't these tools have to update their code anyways to use shadowRoot even if the default was open? If the tools are just using normal DOM accessors and methods, they'd only be getting light DOM anyways. If they've got to update their code to use shadowRoot wouldn't they just add an override so that they could access closed shadow roots?

[trusktr]

@mrmr1993

I want it to be that the application developer retains final creative control, whether the web component developer likes their decisions or not.

The "application developer" is not always a single person. The "application developer" might be a team, and the team will mix match components that the team creates in various UIs. The explicit contract in closed trees is a form of communication between the authors in the team. It encourages collaboration when the external API of a component doesn't satisfy a use case, which is much better than team members haphazardly modifying things and introducing bugs without consulting the original component author. As for 3rd-party components, that is why choice is a freedom. Want open components? Then don't use closed ones. Want to modify a closed 3rd party component? Then fork it or don't use it. More choices means more freedom of design.

[trusktr]

There's also my ComposedTreeProxy idea (#516) which I think might make sense for interaction scripting: if there's something a user can do on screen, then we should also be able to do it by code too (regardless of if a shadow tree is closed or not). Actual users and robot users should be treated equally, but they are not, which is the only downside to closed trees at the moment. Monkey patching to achieve this oversteps the purpose of closed trees, but the proxy idea does not overstep. If a component author puts a button inside a closed tree, and the button appears on screen, then the button is meant to be clicked. If we can't do this with code, then that is a flaw because it means part of the public API (being able to click a button that appears on screen) cannot be done via code but only via actual user interaction. Perhaps another option is that closed trees can be accessed, but when accessed the result is actually a limited proxy (for example, the proxy allows triggering a click, but not the setting of attributes).

[treshugart]

FWIW I predict that if there isn't a method for reading a closed tree, the community will be very quick to override attachShadow() and it will become common practice. If that's acceptable then so be it. If not, then there is a need for a way to access them non-destructively.

[hayatoito]

I am triaging open issues. Let me close this issue since it looks this issue no longer gets much attention.

[trusktr]

Polymer has an idea called FlattenedNodesObserver for traversing a composed tree.