Remove /deep/ usage #2371

ebidel · 2017-05-26T02:56:00Z

Fixes #1913

Also enables shadow dom on the rel="noopener" audit 👊

ebidel · 2017-05-26T03:52:29Z

lighthouse-cli/test/smokehouse/dobetterweb/dbw-expectations.js

@@ -152,7 +152,7 @@ module.exports = [
        score: 0,
        extendedInfo: {
          value: {
-            0: {value: '6,025'},
+            0: {value: '6,037'},


TIL that document.querySelector('html, html /deep/ *') returns different results for shadow dom v0 vs shadow dom v1! I've used it in the past for v0 (e.g. chromestatus returns correct results), but the dbw tester uses shadow dom v1, so we weren't catching all the nodes that we should have been. No idea why v0 and v1 are different, but they deprecated /deep/ before v1 came out. So may be it was buggy with the new API and they never fixed it.

ebidel · 2017-05-26T03:53:15Z

lighthouse-core/gather/gatherers/image-usage.js

@@ -38,7 +39,10 @@ function collectImageElementInfo() {
    };
  }

-  const htmlImages = [...document.querySelectorAll('img')].map(element => {
+  const allElements = getElementsInDocument();


small perf one on the reduced qSA calls :)

wait, what's the perf improvement?

Oh, just that we went from 2 qSA to 1. Before we had a sep query just for document.querySelectorAll('img'). But now, allElements is inclusive of all the img elements on the page. Filtering allElements is probably not much faster, but it's nice to combine the collection calls.

paulirish · 2017-06-06T02:33:37Z

lighthouse-core/gather/driver.js

+   *     True by default.
+   * @return {!Promise<!Array<!Element>>} The found elements, or [], resolved in a promise
+   */
+  getAllNodesInDocument(pierce = true) {


you filter on nodetype and then return our element instance.
and this pierces iframes whereas the dom helper doesnt.

getAllElementsInDocumentAndFrames ?

paulirish · 2017-06-06T02:40:05Z

lighthouse-core/gather/gatherers/image-usage.js

@@ -139,7 +144,12 @@ class ImageUsage extends Gatherer {
      return map;
    }, {});

-    return driver.evaluateAsync(`(${collectImageElementInfo.toString()})()`)
+    const expression = `(function() {
+      ${DOMHelpers.getElementsInDocument.toString()};


took me a moment to figure out what you're doing here.

we should use this library consistently. If this makes the most sense, then in anchors-with-no-rel-noopener.js we should use it similar rather than ${fn.toString()}(selector)

paulirish · 2017-06-06T02:41:29Z

lighthouse-core/lib/dom-helpers.js

+}
+
+module.exports = {
+  getElementsInDocument


since this is only going to be consumed as a string, i think we should toString it at this point. We can append a suffix to help clarify that this is for clientside. perhaps getElementsInDocumentFn?

stramel · 2017-06-06T03:49:52Z

lighthouse-core/gather/driver.js

+  getAllNodesInDocument(pierce = true) {
+    return this.sendCommand('DOM.getFlattenedDocument', {depth: -1, pierce})
+      .then(result => {
+        const elements = result.nodes.filter(node => node.nodeType === 1);


Should this use the Node.ELEMENT_NODE enum here?

Sure, why not. :)

This is driver code. No Node.ELEMENT_NODE :(

stramel · 2017-06-07T04:04:58Z

lighthouse-core/lib/dom-helpers.js

+        results.push(el);
+      }
+      // If the element has a shadow root, dig deeper.
+      if (el.shadowRoot) {


Hmmmm should this be using root here instead of shadowRoot or shadyRoot? Didn't really find any documentation on root.

.shadowRoot is the standard property. root and shadyRoot are Polymer APIs

stramel · 2017-06-07T04:40:12Z

lighthouse-core/gather/driver.js

+   *     True by default.
+   * @return {!Promise<!Array<!Element>>} The found elements, or [], resolved in a promise
+   */
+  getAllNodesInDocument(pierce = true) {


@paulirish @ebidel this is what I had come up with for querying through shadow dom.

queryAllElements(query, includeUserAgentShadowDOM = true) { return this.sendCommand('DOM.performSearch', {query, includeUserAgentShadowDOM}) .then(({searchId, resultCount}) => { if (resultCount === 0) { return []; } return this.sendCommand('DOM.getSearchResults', { searchId, fromIndex: 0, toIndex: resultCount }); }) .then(nodeList => { const elementList = []; nodeList.nodeIds.forEach(nodeId => { if (nodeId !== 0) { elementList.push(new Element({nodeId}, this)); } }); return elementList; }); }

Quick testing was giving me accurate results like the previous html, html /deep/ * => html, html *

stramel · 2017-06-07T04:59:47Z

lighthouse-core/lib/dom-helpers.js

+      }
+    }
+  };
+  _findAllElements(document.querySelectorAll('*'));


Can you support passing in a starting node? Defaulting to document? Should be pretty easy

not sure we need that flexibility quite yet. The method name is getElementsInDocument, so it's obvious that it acts on the entire DOM.

ebidel · 2017-06-15T18:04:50Z

PTAL

ebidel · 2017-06-20T21:27:34Z

@paulirish wdyt?

ebidel · 2017-06-28T19:03:25Z

I will be merging this today if there are no comments :)

brendankenny · 2017-06-28T19:12:04Z

10 other PRs will be happy to hear about our new policy :P

patrickhulce · 2017-06-28T19:39:09Z

lighthouse-core/lib/dom-helpers.js

+}
+
+module.exports = {
+  getElementsInDocumentFn: getElementsInDocument.toString()


would it be too much to call it getElementsInDocumentFnString?

ebidel · 2017-06-28T21:37:16Z

PTAL

patrickhulce

lgtm

⌛️

brendankenny

it's unfortunate not to have a driver-answer to "how do I get all the elements matching a selector, including in shadow trees". Any way we can wrap that up in a usable driver method?

brendankenny · 2017-06-28T21:47:29Z

lighthouse-core/gather/driver.js

+   *     True by default.
+   * @return {!Promise<!Array<!Element>>} The found elements, or [], resolved in a promise
+   */
+  getAllElementsInDocumentAndFrames(pierce = true) {


avoid the boolean trap for now by just not making pierce optional until it's needed?

I was modeling after the DT protocol APIs, which make pierce optional. With the exception that they default to false. We want to default to true.

right, but getAllElementsInDocumentAndFrames(false) is confusing on its own. Don't get all elements?

We could change it somehow to avoid the boolean trap, but I was just suggesting that we don't need to because we don't actually have any uses for not piercing.

If you really think it should stay, though, we should rename somehow (if pierce is false it won't be looking in the "frames" part of getAllElementsInDocumentAndFrames), and maybe make the parameter noPierce so that undefined can just act as false as is usually expected?

In that case, I think we should go back to what I had originally: getAllElementsInDocument(pierce = true).

Closest to the DT protocol for both func and param naming. The docs already explain what pierce does ("pierces through iframes and shadow roots when true"). It doesn't need to be in the method name.

brendankenny · 2017-06-28T21:48:32Z

lighthouse-core/gather/driver.js

@@ -743,6 +743,20 @@ class Driver {
  }

  /**
+   * Returns the flattened list of all DOM nodes within the document.


all DOM element nodes maybe, to be specific?

Think the @return value covers us.

I meant in regards to the "nodes" part...it's only returning nodes with nodeType 1. Could also just be "Returns the flattened list of all DOM elements within the document."

brendankenny · 2017-06-28T21:54:30Z

lighthouse-core/gather/gatherers/image-usage.js

@@ -38,7 +39,10 @@ function collectImageElementInfo() {
    };
  }

-  const htmlImages = [...document.querySelectorAll('img')].map(element => {
+  const allElements = getElementsInDocument();


wait, what's the perf improvement?

ebidel · 2017-06-28T22:16:33Z

it's unfortunate not to have a driver-answer to "how do I get all the elements matching a selector, including in shadow trees". Any way we can wrap that up in a usable driver method?

Agreed. Pavel wasn't a fan of supporting this at the protocol level:
ChromeDevTools/devtools-protocol#19

brendankenny · 2017-06-28T22:32:36Z

Agreed. Pavel wasn't a fan of supporting this at the protocol level

So is there anything we can do on LH's side? I guess one issue would be is it possible for evaluateAsync to send back node IDs compatible with the ones the protocol returns

ebidel commented May 26, 2017

View reviewed changes

paulirish previously requested changes Jun 6, 2017

View reviewed changes

paulirish mentioned this pull request Jun 6, 2017

Audit: input fields have appropriate type & autocomplete attributes #1639

Open

stramel reviewed Jun 6, 2017

View reviewed changes

stramel reviewed Jun 7, 2017

View reviewed changes

ebidel force-pushed the removeshadowdom branch from 4f85bca to 9c849a7 Compare June 15, 2017 18:04

brendankenny assigned paulirish Jun 28, 2017

patrickhulce reviewed Jun 28, 2017

View reviewed changes

ebidel added 3 commits June 28, 2017 14:33

Remove /deep/ usage

e67a3b1

Feedback

e96df2b

getElementsInDocumentFnString

f8103da

ebidel force-pushed the removeshadowdom branch from 9c849a7 to f8103da Compare June 28, 2017 21:36

patrickhulce approved these changes Jun 28, 2017

View reviewed changes

brendankenny reviewed Jun 28, 2017

View reviewed changes

Fix tests. Rename fun back to original

b7443f3

ebidel merged commit 6d862b0 into master Jun 29, 2017

ebidel deleted the removeshadowdom branch June 29, 2017 04:19

Remove /deep/ usage #2371

Remove /deep/ usage #2371

Conversation

ebidel commented May 26, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel Jun 15, 2017 • edited Loading

Choose a reason for hiding this comment

stramel Jun 7, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel commented Jun 15, 2017

ebidel commented Jun 20, 2017

ebidel commented Jun 28, 2017

brendankenny commented Jun 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel commented Jun 28, 2017

patrickhulce left a comment

Choose a reason for hiding this comment

brendankenny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel commented Jun 28, 2017

brendankenny commented Jun 28, 2017

ebidel commented May 26, 2017 •

edited

Loading

ebidel Jun 15, 2017 •

edited

Loading

stramel Jun 7, 2017 •

edited

Loading