ShadowDOM compatibility: Use child-to-parent observation instead of parent-to-child observation for constructing the scene graph from the HTML elements, then observe slot distribution in ShadowDOM roots.

[trusktr]

This will solve the problem of being able to detect when a <lume-node> element is improperly slotted/distributed into a ShadowDOM tree. Without this change (and with respect to elements that are distributed into a ShadowDOM tree), it is not possible to accurately construct our virtual scene graph because children elements observe their parent elements in order to determine how to attach virtual-scene-graph nodes, but closed ShadowDOM trees prohibit children from observing their parents (otherwise the ShadowDOM tree information would leak to the outer world which defeats ShadowDOM encapsulation).

For more background info:

• How can a custom element detect when it is transcluded into a shadow tree (slotchange)? WICG/webcomponents#504
• How to deal with Shadow DOM? aframevr/aframe#1492
• http://stackoverflow.com/questions/37377933

Currently, when a <lume-node> element is attached into the DOM, it checks to see if it's parent element that it was attached to ("connected" to in v1 API terms) is another <lume-node> or <lume-scene> element. If this is not the case, the child <lume-node> element throws an error.

However, this check is not possible when the <lume-node> element has been distributed into a ShadowDOM tree because from the perspective of the distributed <lume-node> element, it's parent is the parent it was attached to outside of the ShadowDOM tree, and it can not get a reference to it's containing <content> element (or <slot> element in v1 API), and therefore it cannot check that the parent of the <content> or <slot> element is another <lume-node> or <lume-scene> element.

To fix the problem we will have parent <lume-node> elements make the observation of their children instead of child <lume-node> elements looking at their parents. Parent <lume-node> elements that are inside of a ShadowDOM tree will have access to their <content> or <slot> element, and will be able to observe what gets distributed into there, and so the parent can throw the error based on the children it observes instead of the child throwing the error based on the parent it observes.

We also need to double check that our behind-the-scenes virtual scene graph is connected based on parent-to-child observation as well, so this way the API will be easier to understand if the flow of observation is the same direction in the virtual scene graph as well as the DOM. It already is.

cc @dmarcos, @cvan, @ngokevin, this will apply to a-frame elements too if they are to be ShadowDOM-compatible. I'm not sure if this is the case already (as far as parent-to-child observation). I know that a-frame isn't considering ShadowDOM yet, so HTML developers will run into this problem with A-Frame at some point when/if they decide to try using A-Frame in ShadowDOM trees.

[trusktr]

The problem with parent-to-child observation of the tree is this is only a partial solution: it only works when the parent is one of LUME's custom elements (<lume-node> or <lume-scene>). If a child <lume-node> element gets distributed into a Shadow tree where the parent that receives the child <lume-node> element is not one of my elements, then there's no way for the child to possibly know that it is distributed improperly if the tree is closed because the child (that lives in the light tree) can not dig into the shadow tree (without non-ideally patching attachShadow) to see if it is distributed to another LUME elements. Plus, it would have to traverse a tree to find all slots, and it would have too much cost.

I really want to be able to throw an error in order to make my API helpful.

We need to find another way. So far, everything would work fine if all Shadow trees were open, but they are closed by default.

[trusktr]

• Waiting on Scene does not need to extend from Node (same with HTML interface). #44 to continue with work in this issue.

[trusktr]

I've decided only to convert to parent-to-child API, and for now not detect the error cases. An app will just silently fail. This may be improved in the future...

[trusktr]

Steps to make custom elements shadow-dom compatible:

• Hijack Element.prototype.attachShadow (or
  Element.prototype.createShadowRoot in v0) so that we can make a Map of
  motor- elements to their shadow roots, so we can always get a reference to
  the element's shadow root even if it is closed. We might not need references
  to the roots, just a boolean to know if the element has a root so we can use
  it in the following logic; Actually, if that's the case, we can just use a
  Set, and existence of an element in the set means it has a root. Done in
  commit bc0bf61.
• When a shadow root is added to a motor element, we should observe
  children of that root (we set this up in the hijacked method). If any
  immediate children of the shadow root are motor-nodes, then we add them as
  children to the host of the shadow root. Note: Shadow roots can't have shadow
  roots, so that's not something we need to worry about. Completed in
  c5b7356.
• We also need to consider that v0 roots can be replaced, in which case the
  imperative Nodes of the old root need to be removed from their imperative
  parent, and they won't ever be added again because the old root won't ever be
  used again (it's not yet possible with current ShadowDOM). We should leave
  the elements in the old root despite removing the imperative connections,
  which is what users will expect (i.e. the elements disappearing due to
  Node.removeChild calls might be confusing). Completed in
  c5b7356.
• We need to distinguish children that are the top of a shadow root from
  children in the light tree (remember that both are considered just "children"
  of the same parent node on the imperative side, so we need a way to
  distinguish them). If a declarative node belongs to another declarative node
  that is a shadow root host (has a shadow root), mark its the node as
  possiblyDistributed (it won't be rendered relative to its original place in
  the DOM tree, it will be rendered relative to its distributed location, or
  not at all if it is not distributed). Mark this notation on all
  previously-existing children of a parent when a shadow root is added to that
  parent, or when children are added to a parent that already contains a root.
  For DOM rendering, keeping track of this won't be useful for anything because
  the HTML engine already does that and traverses the (DOM) render tree itself
  (we don't currently traverse while having only DOM rendering, we only
  construct the tree and pass the transforms to the DOM tree for the HTML
  engine to traverse and render), but when we render WebGL, we will need to
  render only shadow tree nodes and ignore light tree nodes when we traverse
  our own tree. Completed in 1299d13.
• When an element is removed from a parent that is a shadow root host, set
  possiblyDistributed to false. Completed in 1299d13.
• Listen to distribution elements to detect distributed <motor-node>
  elements (in v1 use slotchange on <slot> elements and in v0 use
  MutationObserver on <content> elements) from which to create association
  between shadow parent and shadow child. We've got a
  function to detect v0 vs v1
  shadow roots, so we can determine whether to listen to <slot> elements or
  <content> elements. Complete in 748b5ba,
  but only for v1 slot elements. May support v0 content elements later.
• When a motor element is distributed, mark the shadowParent on the
  element, and add the element to the parent's shadowChildren list. Note that
  the imperative Node will now be referenced in two places: the children list
  of it's normal parent, and the shadowChildren list of its shadowParent.
  Completed in ff3b247.
• When an element is undistributed (detected with slotchange or with
  MutationObserver) remove from shadowChildren, null shadowParent.
  Completed in ff3b247.
• We also need to consider top-of-shadow-root children that are slot
  elements, and watch slotchange on those similarly to how we do in
  child{Con,Discon}nectedCallbacks in order to mark nodes distributed to those
  top-level slots as shadowChildren, similarly to the last two points.
  Completed in f685da2.
• While traversing and calculating world transforms, ignore
  possiblyDistributed nodes (we'll know not to render them). If it was
  distributed, it'll appear in a shadowChildren list deeper in the tree. At
  that point we'll know to calculate it's world transform and to render it.
  Partially completed in bf52ca0, we can
  successfully traverse the flat tree. We'll come back to this when we add
  WebGL...

That's all we need in order to render things in WebGL with shadow-dom
compatibility.

[trusktr]

Completed in bf52ca0, I've decided to support only ShadowDOM-v1. At this point I don't think it makes sense to support Custom Elements v0.

[trusktr]

Re-opened, because when I wrote this issue I was making my own WebGL renderer, so traversing the objects would work (traversing their ShadowDOM composed tree). But now that I'm using Three.js for WebGL, we can't simply traverse our own objects, instead we need to connect Three.js Object3D instances so that they are connected in the form of our composed tree; the Object3D instances are completely unaware of the composed tree. We also need to ensure that connected/disconnected reactions are based on the composed tree, not the light tree.

• Connect Three.js instances in the same shape as the composed tree.

[trusktr]

This was completed not too long ago.