Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: a DocumentFragment whose nodes do not get removed once inserted #736

Open
WebReflection opened this issue Mar 6, 2019 · 21 comments

Comments

@WebReflection
Copy link

@WebReflection WebReflection commented Mar 6, 2019

edit I wouldn't mind Node.DOCUMENT_PERSISTENT_FRAGMENT_NODE name/kind neither

TL;DR

The document fragment is a great primitive to wrap together numerous nodes and append these directly as batch, however a fragment loses all its children as soon as appended, making it's one-off usage limited in those cases where a list of nodes, at a certain position, is meant.

This proposal would like to explore the possibility of a live document fragment that:

  • does not lose its childNodes once appended
  • it's still transparent / unaddressable from CSS
  • it's also transparent for any other node so that it's still possible, as example, to append many <TD> or <TR> through such fragment, and keep a reference for future updates

Why

Both virtual DOM based libraries, such React, as well as direct DOM based one, such as hyperHTML, or lit-html, have been implementing their own version of a persistent fragment in a way or another.

If there was a primitive to directly reference more than a node, through a fragment with a well known position on the DOM, I am pretty sure all libraries would eventually move to adopt such primitive, so that a tag function could handle both <p>1</p> and <p>1</p><p>2</p> without needing to re-invent a similar wheel every single time, and making portability between libraries and frameworks easier than ever: it's just a DOM node!

Example

// either persistent or live
const pf = document.createLiveFragment();

// append zero, one, or any amount of nodes
pf.appendChild(document.createElement('TD'));
pf.appendChild(document.createElement('TD'));

// make it live
const lastTr = document.querySelector('#data tr:last-child');
lastTr.appendChild(pf);

// references are still there
pf.childNodes.length; // 2

// the node is invisible though
lastTr.lastChild === lastTr; // false
lastTr.lastChild === lastTr.childNodes[1]; // true

There should be no way to interfere with CSS and/or selectors, the fragment is either referenced somewhere else or it won't exist for the DOM.

How

The way hyper/lit-html are doing this is by abusing comment nodes as boundaries of these virtual fragments. The itchy part of these libraries is mostly represented by these virtual fragments, 'cause it's obvious if the primitive proposed here would exists, these libraries would've used it instead (happy to be corrected, but at least I would never create my own virtual fragment if I could use something else).

The way this could be implemented, is by weakly referencing nodes to such fragment only if this is held in memory.

// example implementation of the live fragment
const references = new WeakMap;
class LiveFragment extends DocumentFragment {
  #childNodes = [];
  #appendChild = node => {
    this.#removeChild(node);
    this.#childNodes.push(node);
  };
  #removeChild = node => {
    const i = this.#childNodes.indexOf(node);
    if (-1 < i)
      this.#childNodes.splice(i, 1);
  };
  appendChild(node) {
    this.#appendChild(node);
    return super.appendChild(reference.call(this, node));
  }
  append(...nodes) {
    nodes.forEach(this.#appendChild);
    return super.append(...nodes.map(reference, this));
  }
  removeChild(node) {
    this.#removeChild(node);
    references.delete(node);
    return super.removeChild(node);
  }
  // the value of this LiveFragment when moved around
  valueOf() {
    this.append(...this.#childNodes);
    return this;
  }
}
function reference(node) {
  references.set(node, this);
  return node;
}

// amend on the appendChild standard
const {appendChild} = Node.prototype;
Node.prototype.appendChild = function (node) {
  appendChild.call(this, asFragment(node));
};

const {append} = Element.prototype;
Element.prototype.append = function (...nodes) {
  append.apply(this, nodes.map(asFragment));
};

function asFragment(node) {
  return node instanceof LiveFragment ?
    // it could be just node.valueOf() for everything
    // explicit here for explanation sake
    node.valueOf() :
    node;
}

Possible F.A.Q. Answers

  • if a node is manually appended somewhere it's fine. But as soon as the LiveFragment owner will append such fragment again, that node would be moved back (nodes ownership by creator)
  • if the LiveFragment has no references, then nothing changes from a fragment
  • a live fragment always exposes its childNodes as immutable, as it is for regular fragments
  • everything is the same, except that live fragments creators, owner of their live fragment content, can use this primitive instead of polluting the DOM with comments
  • it is always possible for DOM engines to know if a node below to a fragment, as long as this is still referenced somewhere. If unnecessary, due forced re-append on valueOf(), the references part can be ignored

Thanks in advance for eventually considering this, happy to answer any possible question.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 6, 2019

FWIW this is apparently a very demanded feature from the Web Developers community

Previously on a similar proposal: https://discourse.wicg.io/t/proposal-live-fragments/2507

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 6, 2019

As simplified approach, and for demo sake, I'll leave a playground that works out of the box in Chrome Canary (but not yet in Code Pen, however I've informed them about it)
https://codepen.io/WebReflection/pen/moRQRV?editors=0010

Now I'll wait for any outcome 👋

@annevk annevk transferred this issue from whatwg/html Mar 7, 2019
@annevk annevk changed the title Proposal: Node.DOCUMENT_LIVE_FRAGMENT_NODE Proposal: a DocumentFragment whose nodes do not get removed onse inserted Mar 7, 2019
@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 7, 2019

@annevk just FYI I think there's a typo in the title: onse => once

also, if I might ask, what does the label "needs implementer interest" mean?

What can I do to move this forward? Should it be me the implementer?

Thanks.

@annevk annevk changed the title Proposal: a DocumentFragment whose nodes do not get removed onse inserted Proposal: a DocumentFragment whose nodes do not get removed once inserted Mar 7, 2019
@annevk

This comment has been minimized.

Copy link
Member

@annevk annevk commented Mar 7, 2019

Do https://whatwg.org/working-mode#changes and https://whatwg.org/faq#adding-new-features help?

I suspect we'll need something like this to build templating on top of.

cc @rniwa @justinfagnani @wycats

@developit

This comment has been minimized.

Copy link

@developit developit commented Mar 7, 2019

This is interesting. Other names I've seen used to casually refer to something along these lines are "Persistent Fragment" (given that it doesn't empty when appended) and just "Range Fragment".

Example: https://discourse.wicg.io/t/proposal-fragments/2312

@justinfagnani

This comment has been minimized.

Copy link

@justinfagnani justinfagnani commented Mar 7, 2019

Edit: I didn't see that in this version of the idea the fragment doesn't live in the tree. I think that makes it functionally equivalent to NodePart from TemplateInstantiation, or a wrapper around StaticRange, and addresses most of the issue below, which were based on other variations of the idea I'm familiar with.

@annevk I think the Template Instantiation Part interface is essentially this, without being a Node living in the childNodes list. display: contents also covers some similar use cases.

One concern with a new node type is that much existing tree traversal code will not know how to handle it, so a live fragment and its children will likely be skipped. Depending on where such live fragments are intended to be used this may or may not be a problem. There is a lot of code out there that assumes that only Elements can contain other Elements.

Another concern is that right now I believe that ever Node subclass reachable by tree-walking a document is serializable to HTML (including <template> and it's unique innerHTML implementation). The fact that DocumentFragment cannot be placed into a childNodes list preserves this. I'm not sure how much this matters, but this would be the first node type in the tree (but not tree-of-trees) that can't survive a serialize/parse round-trip.

We'd also have to consider how other APIs work. Do live fragments show up on event.path? Can children of a live fragment be slotted into the fragment's parent's ShadowRoot? etc...

I'm not sure if the issues are insurmountable, but I've been working on Template Instantiation with the theory that the least disruption will be caused with a new non-Node interface that lives outside the tree, like Range. Then all existing tree processing code will work as-is.

@rniwa

This comment has been minimized.

Copy link

@rniwa rniwa commented Mar 10, 2019

Yeah, we've definitely considered this approach before making the template instantiation proposal but fundamentally, all we need is tracking where the inserted contents need to be. I don't think there is any reason to create a new node type and keep it in the DOM if we can avoid it.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 10, 2019

Nothing is kept in the DOM. It's a fragment that acts like a fragment.
I'm working on a better playground that shows the idea fully polyfilled so please wait for it to be online and evaluable before closing this, thanks.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 11, 2019

So, I've uploaded the previously mentioned polyfill, which should have 100% code coverage.
https://github.com/WebReflection/document-persistent-fragment

There is a live test page too.

the what

The idea is to have a 1:1 DocumentFragment alter-ego that doesn't lose its nodes.

The fragment is exposed only to the owner, so that there is no way to retrieve it from third parts, unless passed around, and there's nothing live on the DOM, if not, eventually its child nodes.

The proposal exposes to the owner common nodes methods based on its content.

As example, dpf.previousSibling would return the dpf.firstChild.previousSibling if there is a firstChild. If the dpf.isConnected is false, the result is null.

The DPF (in short) has only one extra method, compared to DocumentFragment, which is dpf.remove(). Such method doesn't need much explanation: it removes all nodes from the DOM.

All operations performed through the DPF are reflected live on the document, and while this might be just a stretch goal, it is super easy and nice to simply update an owned reference and see everything changing live.

Nodes ownership

It is possible to grab random nodes and destroy these, or change these, affecting indirectly the content owned by the DPF instance, but it's always been possible to be obtrusive on the DOM and destroy third parts libraries so I think this shouldn't be concern.

However, I could implement the WeakMap that relate each node to a specific DPF instance in a way that it's not possible to move a node owned by a DPF into another DPF, or even perform any action on the DOM through elements that are not the DPF owner.

This, however, would introduce an ownership concept that is too different from what we've used so far, but I believe this proposal is for all libraries that need such persistent fragment, and that owns their own nodes, libraries that are currently somehow already breaking things if 3rd parts obtrusive libraries destroy, or manipulate, DOM nodes in the wild.

As summary

The fact, beside some Safari glitch I'm sure I can solve, this whole proposal can be already polyfilled, and the fact browsers have a way to optimize it and make it blazing fast, should be considered as a plus, 'cause instantly adoptable by the community, so that we can have quick feedbacks of how much this is welcomed or needed out there.

Please don't hesitate to file issues there or ask me more here before discarding this proposal.

Thank You.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 11, 2019

@rniwa FYI I've filed the bug that makes current polyfill not usable * in WebKit/Safari

edit I've fixed the current polyfill with a workaround after a feature detection, so this can work on Safari/WebKit too 👋

@rniwa

This comment has been minimized.

Copy link

@rniwa rniwa commented Mar 11, 2019

Nothing is kept in the DOM. It's a fragment that acts like a fragment.

Then what you created is indistinguishable from NodeTemplatePart we proposed. It's just a matter of syntax / naming differences.

@thysultan

This comment has been minimized.

Copy link

@thysultan thysultan commented Mar 11, 2019

@rniwa NodeTemplatePart seems to be tightly linked to shadow dom and template instantiation; how would you use NodeTemplatePart to replicate the proposed?

@rniwa

This comment has been minimized.

Copy link

@rniwa rniwa commented Mar 11, 2019

@rniwa NodeTemplatePart seems to be tightly linked to shadow dom and template instantiation

It's nothing to do with Shadow DOM.

how would you use NodeTemplatePart to replicate the proposed?

In the latest iterations of the proposal @justinfagnani at Google and we're working on, NoteTemplatePart is a thing that could be used without any template although we probably need to rename it to something else.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 12, 2019

@rniwa I've no idea what is this NodeTemplatePart and I cannot find anything online, however, if you would read at least the test you'll see this proposal is literally nothing new, it's fully based on current standard, backward compatible, and polyfillable.

It's a document-fragment at all effects, it's indeed inheriting the same constructor, but it works transparently and only if the owner/creator keeps a reference around.

If this is exactly what this NodeTemplatePart does, can you please point me at it's specification or API?

Thanks.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 12, 2019

@justinfagnani

Edit: I didn't see that in this version of the idea the fragment doesn't live in the tree.

would you mind amending/canceling that comment since nothing in there is relevant to this proposal, so that people don't get distracted by concerns that are not part of this proposal?

links to the solutions previously discussed would be more than welcome too.

Thanks.

@annevk

This comment has been minimized.

Copy link
Member

@annevk annevk commented Mar 14, 2019

The context you're missing is https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md and some F2F discussion (probably somewhere in minutes linked from issues in that repository) that encouraged making the parts there true primitives instead of tightly coupled with templating.

(Also, please try to consolidate your replies a bit. Each new comment triggers a new notification for some and there's over a hundred people watching this repository. When in doubt, edit an existing reply.)

There's another meeting coming up, and I hope @justinfagnani and @rniwa can make the current iteration a bit more concrete by then, as referencing it in this issue as if it's a thing everyone should be aware of is a lil weird.

@ryansolid

This comment has been minimized.

Copy link

@ryansolid ryansolid commented Mar 19, 2019

The context you're missing is https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md and some F2F discussion (probably somewhere in minutes linked from issues in that repository) that encouraged making the parts there true primitives instead of tightly coupled with templating.

Thank you, I'm glad I read the second part of that comment about true primitives. I had read that proposal before and admittedly a little terrified that the DOM spec would include such an opinionated implementation. That spec reads like designing a render framework. I mean I'm sure we could do worse, but I'm encouraged to know that simpler proposals are under consideration.

What I like about this proposal is the transparent use with Node API's appendChild/removeChild since it can be treated interchangeably with actual DOM nodes when say returned from some sort of Template instantiation.

I think ownership is the challenge with a lot of value comes from having a clear single ancestor(whether single node or fragment). It lets JS context be tied to specific parent nodes without making more elements in the DOM. But by their very nature I don't see how you'd avoid nesting. Like a loop over these fragments with a top level conditional in it that also returns a fragment. In the end the DOM structure would be pretty flat but you'd have persistent fragments in persistent fragments. Since they aren't part of the actual rendered tree it becomes harder to understand what falls under each since nested dynamic content in this case could change what is in the top fragment. I would absolutely love to see a solution in this space having spent a lot of time figuring out tricks to accomplish similar things. Everyone writing a render library where they don't virtualize the tree hits this issue sooner or later.

@WebReflection

This comment has been minimized.

Copy link
Author

@WebReflection WebReflection commented Mar 19, 2019

you'd have persistent fragments in persistent fragments

which is fine, and since appending a fragment to itself breaks, there's nothing different on the DOM.

Current implementation / proposal allows shared nodes between fragments, which is the same as creating a fragment on the fly and append any node found in the wild: nothing stops your from doing that, everything works, no error thrown.

The current idea is that keeping it simple is the only way to quickly move forward, while any ownership concept would require bigger, non backward compatible, changes.

Libraries and frameworks authors won't worry about that anyway, 'cause they are the one creating nodes too, and they are those virtualizing thee in trees.

Having a mechanism to move N nodes at once, accordingly with any DPF appended live, it's also a feature that would simplify state-machine driven UIs.

Last, but not least, hyperHTML has this primitive since long time and it works already, but it doesn't play super nice with the DOM if thrown there as is, and it requires special treatment when used right away.

This proposal would cover that case can much more.

@Jamesernator

This comment has been minimized.

Copy link

@Jamesernator Jamesernator commented Apr 11, 2019

I'm not sure how much this matters, but this would be the first node type in the tree (but not tree-of-trees) that can't survive a serialize/parse round-trip.

Actually CData sections and processing instruction nodes are simply completely broken in text/html because they don't deserialize (instead they just become text nodes) but do serialize. Not that anyone really cares about them.

@jhpratt

This comment has been minimized.

Copy link

@jhpratt jhpratt commented Jun 29, 2019

Is there any continued interest in this proposal? Would a persistent fragment have access to some DOM methods like replaceWith()? Those could be heavily optimized in situations like element reordering.

@ryansolid

This comment has been minimized.

Copy link

@ryansolid ryansolid commented Jul 3, 2019

I like this proposal a lot more after sitting with it a bit longer. At first I was thinking this was about library code managing the moving and managing of ranges of elements, but this is more. This helps with giving the users of said libraries the equivalent of React JSX Fragments. Like consider:

const div = html`<div></div>`
const frag = html`<div></div><div></div>`

// further down
const view = html`<div>${ condition ? div : frag }</div>`

I had a user ask why the div always worked, but the second time they attached the fragment why did it not render anything. The answer of course was that div.appendChild(frag) removed the childNodes from the fragment so on the second attach there were no childNodes to append. Now one could argue I could make frag an array (slice the childNodes) but that doesn't handle dynamic top level changes in the fragment.

In a library based on KVO Observables top level dynamic updates that execute independently of top down reconciliation having a standardized DOM API that works the same whether attached or not is hugely helpful in this scenarios.

Beyond that all these libraries have a similar concept. Being just a DOM node works very consistently with the whole

// tagged templates
const el = html`______` //or
// jsx
const el = <______ />

way of approaching rendering which has been gaining steam (looking at the API's on the top end of performance in the JS Frameworks Benchmark). More and more non-virtual DOM libraries are picking this approach and showing it is performant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
9 participants
You can’t perform that action at this time.