New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rehydration #549

Merged
merged 15 commits into from Jun 21, 2017

Conversation

Projects
None yet
9 participants
@wycats
Contributor

wycats commented Jun 15, 2017

This commit introduces a "rehydration" mode to the Glimmer VM.

-- What is Rehydration?

The rehydration feature means that the Glimmer VM can take a DOM tree
created using Server Side Rendering (SSR) and use it as the starting
point for the append pass.

Rehydration is optimized for server-rendered DOMs that will not need
many changes in the client, but it is also capable of making targeted
repairs to the DOM:

  • if a single text node's value changes between the server and the
    client, the text node itself is repaired.
  • if the contents of a {{{}}} (or SafeString) change, only the
    bounds of that fragment of HTML are blown away and recreated.
  • if a mismatch is found, only the remainder of the current element
    is cleared.

What this means in practice is that rehydration repairs problems
in a relatively targeted way, and doesn't blows away more content
than the contents of the current element when a mismatch occurs.

Near-term work will narrow the scope of repairs further, so that
mismatches inside of blocks only clear out the remainder of the
content of the block.

-- What Changes Were Needed?

Previously, code in the append pass was permitted to make DOM
changes at any point, so long as they made the changes through
the stateless "DOM Helper" abstraction, which ensured that
code didn't inadvertantly rely upon environment-specific
behaviors. This includes both browser quirks (including the
most recent version of Safari), and restricting Glimmer to
the subset of DOM used in SimpleDOM, the library that is
responsible for emulating the DOM in our current Server
Side Rendering implementation.

Rehydration works by replacing attempts to create DOM nodes
with a check for whether the node that Glimmer is trying to
create is already present. It maintains a "cursor" against
the unhydrated DOM, and when Glimmer attempts to create an
element, it first checks to see whether the candidate node
matches the node that Glimmer is trying to create.

For example, if Glimmer wants to create a text node,
the rehydrator checks to see if the node under the cursor
is a text node. If it is a text node, it updates its
contents if necessary. If not, it begins a coarser-grained
repair (which, as described above, is never worse than
clearing out the rest of the DOM for the current element).

This requires that the core DOM operations not only go
through a stateless abstraction (DOM Helper), but also
have a choke-point through a stateful builder that can
maintain the candidate cursor and initiate repairs.

Most of this commit restructures code so that DOM operations during
the append pass go throug the central ElementBuilder in all cases.

Part of this process involved restructuring the code that manages
dynamic content and dynamic attributes to simplify them and make
them a better fit for ensuring that all DOM operations in the
append pass go through the ElementBuilder.

-- Serialize Mode

This commit also introduces a dedicated "serialize" mode that
server-side renderers should use to generate the DOM.

The serialize mode is reponsible for inserting additional markers
into the DOM to serialize boundaries that otherwise serialize
incorrectly.

For example, it inserts a comment node between two adjacent
text nodes so that they remain separated when rehydrating.

The serialize mode does not prescribe a DOM implementation;
any DOM that is compatible with the well-defined SimpleDOM
subset will work (including a real browser's DOM, JSDOM,
or SimpleDOM); it just ensures that the created DOM will
round-trip through a string of HTML.

Importantly, the rehydrator mode assumes that the server-
provided DOM was created in serialize mode.

-- Future Work: Attributes and Properties

Today, when you write something like <div title={{value}}>,
Glimmer attempts to use DOM properties if possible. This
works, more or less, because the DOM automatically reflects
properties to attributes in most cases.

This means that if you write <img src={{url}} /> in a template,
we will set the src property, which the DOM automatically
reflects onto the src attribute.

Using properties where possible as opposed to attributes in
all cases addresses a number of use cases. For example,

  • <select value={{someValue}}> will select the correct
    <option> automatically
  • <div onclick={{action foo}}> "just works" to set
    event handlers.
  • <textarea value={{someValue}}> updates the inside of
    the textarea instead of requiring you to deal with
    the text node contents of the textarea.

We intend to move to all-attributes-all-the-time in the future, but
we need to address these use-cases. There are various approaches
under consideration (including a few special-cases, element
modifiers, and an alternate sigil), but they are out of scope
for this PR.

In order to address this gap for now, this PR works around the slight
semantic inconsistency. When the rehydrator pushes an element, it
snapshots the list of all attributes that are already present on
the element as a list of candidates for attribute removal. When
Glimmer attempts to set an attribute or property on the element,
the rehydrator removes it from the list of candidates for attribute
removal.

When an element's attributes are "flushed" (a step in the Glimmer
VM), the rehydrator removes any attributes that it didn't see during
the append step.

This should work in the vast majority of cases because:

First, it doesn't stop properties from being set, so any case where
properties were set before continue to be set now.

Second, if an attribute was present and the same-named property was
set in the client, the attribute will remain in the DOM. This
can only go wrong if the application was relying on a property
being set that didn't set the same-named attribute. This
is very rare; one example might be if an app was relying on using
form.reset() to reset a field back to '', relying on the
fact that <input value={{someValue}}> doesn't actually
set the value attribute.

This is very unlikely because empirically very few people use
.reset() in Ember or Glimmer, which is why using a property
instead of an attribute here worked in the first place
.

In short, if someone is relying on the fact that an element's
attribute actually sets a property also relies on the fact that
it doesn't set an attribute,
the current workaround will repair
inconsistently. This should be very rare and something we can
address next.

@@ -125,7 +125,7 @@ export namespace DOM {
return this.document.createElementNS(namespace, tag);
}
setAttribute(element: Element, name: string, value: string, namespace?: string) {
setAttribute(element: Element, name: string, value: string, namespace?: Option<string>) {

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

namespace : Option<string> = null?

// import {
// defaultManagers,
// AttributeManager
// } from './dom/attribute-managers';

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

✂️

@@ -455,8 +456,8 @@ export abstract class Environment {
transaction.commit();
}
attributeFor(element: Simple.Element, attr: string, isTrusting: boolean, namespace?: string): AttributeManager {
return defaultManagers(element, attr, isTrusting, namespace === undefined ? null : namespace);
attributeFor(element: Simple.Element, attr: string, isTrusting: boolean, namespace: Option<string>): DynamicAttributeFactory {

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

namespace: Options<string> = null for consistency?

buffer.push(statement);
this.state = LayoutState.AfterFlush;
return;
}

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

✂️ this and modify the resulting failing test

this.processStatement(attrs[i], buffer);
}
this.processStatement([ Ops.FlushElement ], buffer);

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

we might want to extract these into const since they just get copied into another array anyway

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

(there are a bunch in this file)

normalized = value;
} else {
normalized = String(value);
}

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

normalizeStringValue?

__appendHTML(html: string): Bounds {
let first = this.__appendComment('%glimmer%');
super.__appendHTML(html);
let last = this.__appendComment('%glimmer%');

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

This probably wants to be a matching pair, like <!-- {{{ --> and <!-- }}} --> perhaps. If we think it's not important to have a matching pair for this case then an empty comment is just fine.

return bounds(this.element, first, last);
}
__appendText(string: string): Simple.Text {

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

Does not handle empty text nodes

let current = currentNode(this);
if (current && current.nodeType === Node.TEXT_NODE) {
this.__appendComment('%sep%');

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

Could just use an empty comment here

@@ -16,7 +16,7 @@
"noImplicitAny": true,
"suppressImplicitAnyIndexErrors": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
// "noUnusedParameters": true,

This comment has been minimized.

@chancancode

chancancode Jun 15, 2017

Contributor

revert

@jeffwhelpley

This comment has been minimized.

jeffwhelpley commented Jun 16, 2017

We have considered something similar for Angular in the past and are still experimenting with a couple diff options. One thing I am curious about. Do you have performance metrics for larger pages comparing full view re-rendering vs this rehydration approach?

@gdi2290

This comment has been minimized.

gdi2290 commented Jun 16, 2017

great write up 👍

@sclatter

This comment has been minimized.

sclatter commented Jun 17, 2017

Super, @wycats !

wycats and others added some commits Jun 14, 2017

Rehydration
This commit introduces a "rehydration" mode to the Glimmer VM.

-- What is Rehydration?

The rehydration feature means that the Glimmer VM can take a DOM tree
created using Server Side Rendering (SSR) and use it as the starting
point for the append pass.

Rehydration is optimized for server-rendered DOMs that will not need
many changes in the client, but it is also capable of making targeted
repairs to the DOM:

- if a single text node's value changes between the server and the
  client, the text node itself is repaired.
- if the contents of a `{{{}}}` (or SafeString) change, only the
  bounds of that fragment of HTML are blown away and recreated.
- if a mismatch is found, only the remainder of the current element
  is cleared.

What this means in practice is that rehydration repairs problems
in a relatively targeted way, and doesn't blows away more content
than the contents of the current element when a mismatch occurs.

Near-term work will narrow the scope of repairs further, so that
mismatches inside of blocks only clear out the remainder of the
content of the block.

-- What Changes Were Needed?

Previously, code in the append pass was permitted to make DOM
changes at any point, so long as they made the changes through
the stateless "DOM Helper" abstraction, which ensured that
code didn't inadvertantly rely upon environment-specific
behaviors. This includes both browser quirks (including the
most recent version of Safari), and restricting Glimmer to
the subset of DOM used in `SimpleDOM`, the library that is
responsible for emulating the DOM in our current Server
Side Rendering implementation.

Rehydration works by replacing attempts to create DOM nodes
with a check for whether the node that Glimmer is trying to
create is already present. It maintains a "cursor" against
the unhydrated DOM, and when Glimmer attempts to create an
element, it first checks to see whether the candidate node
matches the node that Glimmer is trying to create.

For example, if Glimmer wants to create a text node,
the rehydrator checks to see if the node under the cursor
is a text node. If it is a text node, it updates its
contents if necessary. If not, it begins a coarser-grained
repair (which, as described above, is never worse than
clearing out the rest of the DOM for the current element).

This requires that the core DOM operations not only go
through a stateless abstraction (DOM Helper), but also
have a choke-point through a stateful builder that can
maintain the candidate cursor and initiate repairs.

Most of this commit restructures code so that DOM operations during
the append pass go throug the central ElementBuilder in all cases.

Part of this process involved restructuring the code that manages
dynamic content and dynamic attributes to simplify them and make
them a better fit for ensuring that all DOM operations in the
append pass go through the ElementBuilder.

-- Serialize Mode

This commit also introduces a dedicated "serialize" mode that
server-side renderers should use to generate the DOM.

The serialize mode is reponsible for inserting additional markers
into the DOM to serialize boundaries that otherwise serialize
incorrectly.

For example, it inserts a comment node between two adjacent
text nodes so that they remain separated when rehydrating.

The serialize mode does not prescribe a DOM implementation;
any DOM that is compatible with the well-defined SimpleDOM
subset will work (including a real browser's DOM, JSDOM,
or SimpleDOM); it just ensures that the created DOM will
round-trip through a string of HTML.

Importantly, the rehydrator mode assumes that the server-
provided DOM was created in serialize mode.

-- Future Work: Attributes and Properties

Today, when you write something like `<div title={{value}}>`,
Glimmer attempts to use DOM properties if possible. This
works, more or less, because the DOM automatically reflects
properties to attributes in most cases.

This means that if you write `<img src={{url}} />` in a template,
we will set the `src` property, which the DOM automatically
reflects onto the `src` attribute.

Using properties where possible as opposed to attributes in
all cases addresses a number of use cases. For example,

- `<select value={{someValue}}>` will select the correct
  `<option>` automatically
- `<div onclick={{action foo}}>` "just works" to set
  event handlers.
- `<textarea value={{someValue}}>` updates the inside of
  the textarea instead of requiring you to deal with
  the text node contents of the textarea.

We intend to move to all-attributes-all-the-time in the future, but
we need to address these use-cases. There are various approaches
under consideration (including a few special-cases, element
modifiers, and an alternate sigil), but they are out of scope
for this PR.

In order to address this gap for now, this PR works around the slight
semantic inconsistency. When the rehydrator pushes an element, it
snapshots the list of all attributes that are already present on
the element as a list of candidates for attribute removal. When
Glimmer attempts to set an attribute **or property** on the element,
the rehydrator removes it from the list of candidates for attribute
removal.

When an element's attributes are "flushed" (a step in the Glimmer
VM), the rehydrator removes any attributes that it didn't see during
the append step.

This should work in the vast majority of cases because:

First, it doesn't stop properties from being set, so any case where
properties were set before continue to be set now.

Second, if an attribute was present and the same-named property was
set in the client, the attribute will remain in the DOM. This
can only go wrong if the application was relying on a property
being set that **didn't** set the same-named attribute. This
is very rare; one example might be if an app was relying on using
`form.reset()` to reset a field back to `''`, relying on the
fact that `<input value={{someValue}}>` doesn't **actually**
set the value attribute.

This is very unlikely because empirically very few people use
`.reset()` in Ember or Glimmer, which is **why using a property
instead of an attribute here worked in the first place**.

In short, if someone is relying on the fact that an element's
attribute actually sets a property also **relies on the fact that
it doesn't set an attribute,** the current workaround will repair
inconsistently. This should be very rare and something we can
address next.
Revert "Run npm i again"
This reverts commit 96584b4.

@chadhietala chadhietala force-pushed the rehydration branch from 1325aec to b1ef501 Jun 19, 2017

return callback();
}
protected assertStableNodes({ except: _except }: { except: Set<Node> | Node | Node[] } = { except: new Set() }) {

This comment has been minimized.

@rwjblue

rwjblue Jun 19, 2017

Member

We still support browsers without native Set (and we do not require polyfills). We need to replace this with something else...

This comment has been minimized.

@chadhietala

chadhietala Jun 19, 2017

Member

I have this fixed locally. All we need to do is uniq the array.

chadhietala added some commits Jun 19, 2017

@chadhietala chadhietala force-pushed the rehydration branch from ed47f45 to bb21d74 Jun 20, 2017

chadhietala added some commits Jun 21, 2017

@chadhietala chadhietala force-pushed the rehydration branch from b58d4d7 to b64e1b6 Jun 21, 2017

@chadhietala chadhietala merged commit cdd69e2 into master Jun 21, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@ming-codes

This comment has been minimized.

Contributor

ming-codes commented Aug 3, 2017

How will this affect DOM nodes that are moved/reordered without Glimmer knowing? Will rehydration just blow that away and rebuild?

@locks locks deleted the rehydration branch Aug 3, 2017

@chadhietala

This comment has been minimized.

Member

chadhietala commented Aug 4, 2017

When we are in rehydration mode we scan over top of the SSR'd HTML. When we detect something is not correct we clear to the end of the current bounds and client-side render it. This means that we only throw out subsections of the SSR'd page when they don't match as opposed to chucking the whole document out and starting over.

@jeffwhelpley

This comment has been minimized.

jeffwhelpley commented Aug 4, 2017

@chadhietala how is hydration matching up which node in the server view corresponds to a particular client side component? Is it based off position in the DOM tree?

@wycats

This comment has been minimized.

Contributor

wycats commented Aug 4, 2017

@jeffwhelpley it's based on position in the DOM + some additional markers to track block start and end.

In Glimmer, a template like this:

<div>{{hello}}</div>

<some-component />

<some-other-component>{{contents}}</some-other-component>

is actually highly static. The output from the server-side serializer will place block boundaries around the dynamic parts that can contain more than one node (like components), but the nodes themselves cannot move around. As a result, "position in DOM" is a relatively precise way to identify nodes in Glimmer.

@gdi2290

This comment has been minimized.

gdi2290 commented Aug 4, 2017

I see, so similar to Resource Hints you're providing hints to the client compiler from the server compiler to create the correct structure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment