Use new parse5 streaming interface #1316

Sebmaster · 2015-12-09T15:56:15Z

ToDo:

Add stylesheet removal test
Add test for link styleSheets insertion timing (before / after load)
Test that stylesheets are already added/removed (in styleSheets) when dom mutation events are emitted
Fix script insertion
Add test for incomplete tag document.writes

Sebmaster · 2015-12-09T15:57:39Z

This doesn't use the new JS capabilities parse5 got, but gives us the new location info that was added.

Sebmaster · 2015-12-09T16:00:23Z

lib/jsdom/browser/htmltodom.js

@@ -237,7 +267,26 @@ function setChild(core, parent, node) {
    return null;
  }

-  newNode[locationInfo] = node.__location;
+  if (node.__location) {
+    newNode[locationInfo] = {


Not too sure about this. This is mainly needed for backwards compat because start/end got renamed to (start|end)Offset. We could just alias them, or major ourselves...

domenic · 2015-12-09T16:11:41Z

I'd like to be a bit more ambitious than this. We should do a major version bump and:

Stop using parse5's htmlparser2 adapter, thus solving problems mentioned in Things that are awkward in jsdom right now in 1.5 inikulin/parse5#78. This should remove a lot of the code where we construct the tree manually; if we just provide a nice parse adapter to parse5, it should be able to construct it for us.
Remove support for all parsers except sax and parse5. Figure out the best way to change the parser option so that if statements are minimal. If people want to use other parsers they must ape the parse5 or sax interface. E.g. maybe we require HTML parsers to have parse, parseFragment, serialize, serializeFragment, and ParserStream?? We can remove the parser option until we switch parse5's new interface, if that makes it easier.
Clean up dead code where we are patching up old parsers, e.g. parseDoctype.

Sebmaster · 2015-12-09T16:14:36Z

Sounds good to me, will do that.

Sebmaster · 2015-12-12T05:16:17Z

Finished up our document adapter, however there're a few issues there, one of them being that we don't handle scripts correctly now, not sure if there are others as well. I think parse5 creates a script, appends it to the dom and inserts text after, but we don't notice the appended text. The HTML spec describes how to handle parser inserted scripts correctly - I hope that's do-able with the jsdom-parse5 interface(s).

sykaeh · 2016-01-14T16:30:14Z

Any updates on this?

Sebmaster · 2016-01-15T03:27:58Z

@sykaeh It's the next thing I'll tackle, the node rewrite got in the way, but I'll have to re-implement scripts for this to work and that (or a different part afterwards) might reveal some more issues I haven't thought of yet.

inikulin · 2016-01-15T08:09:04Z

lib/jsdom/browser/htmltodom.js

-      setChild(this.core, element, parsed[i]);
+      const instance = new this.parser.ParserStream({
+        treeAdapter: adapter(element),
+        locationInfo: true


I would recommend to not enable locationInfo by default. It has significant impact on performance and since it's not commonly used, I suggest to make it optional and disabled by default in jsdom.

domenic · 2016-01-15T17:44:30Z

Wishlist: allow "invoking the fragment parsing algorithm with text as markup, and context as the context element" from other parts of jsdom. See #1355. Maybe requires parse5 support, haven't checked.

Sebmaster · 2016-01-26T13:15:29Z

Our resource loader is currently killing my progress - it's way too tightly integrated with our delay-load-event logic, but is also the only place which contains our logic on how to correctly resolve urls to either files or urls and how to handle a custom fetch callback :(

Will need to work around it or clean it up...

During script execution, the document wasn't actually loaded yet, but our old parser implementation loaded the whole document before executing scripts.

MeirionHughes · 2016-08-18T07:31:18Z

Any progress?

Fixes #1778. This changes the public API in that the nodeLocation() functions now return differently-structured objects. parse5 has also removed support for serialization of obsolete doctypes. #1316 is a more ambitious attempt to use parse5's new streaming interface to get correct script execution semantics, but is still not ready; this is just a minimal upgrade to the latest parse5 version.

TimothyGu · 2018-01-16T07:38:53Z

An update: parts of the cleanup is now being developed in #2098. Resource loader is still there however, for now.

This changes the HTML parsing infrastructure, in particular how we interface with parse5. The original motivation for this change was increased performance; the new benchmark, which appends 65535 <tr>s, gets a speedup of ~3x from these changes. Other user-visible changes to script evaluation might be present. In particular, the tests revealed that one document.write-related test that used to somehow pass now fails, while another one succeeds (at least on Node 8). In most cases, however, behavior should be the same. While here, we also changed style sheets to reevaluate their rules and update styleEl.sheet or linkEl.sheet appropriately when their child text content changed. Behind the scenes details: - We now use a parse5 tree adapter directly for parsing, instead of using the htmlparser2 adapter layer (which had an O(n) insertion cost for new siblings) - SAX XML parsing code has been simplified by no longer being shared with parse5/htmlparser2 parsing code. - Nodes no longer have a reference to the "core" god-object. This was only used in a couple places, and was error prone because this reference was not available in cases such as document nodes created via the Document constructor. This removes a lot of code that threaded the object throughout everything. - We continued to use hacky workarounds for script evaluation, during parsing and elsewhere. Perhaps one day, inspired by #1316 and #1920, we can fix these.

Sebmaster mentioned this pull request Dec 9, 2015

Update parse5 to 2.0.0 to get line/col numbers in LocationInfo #1315

Closed

Sebmaster reviewed Dec 9, 2015
View reviewed changes

Sebmaster force-pushed the feature/new-parse5 branch from 74e4e49 to e2c9f17 Compare December 12, 2015 04:31

Sebmaster force-pushed the feature/new-parse5 branch 7 times, most recently from 046375f to 5803c7e Compare December 13, 2015 20:24

Sebmaster force-pushed the feature/new-parse5 branch 2 times, most recently from 4654671 to b7b7e74 Compare January 15, 2016 01:55

inikulin reviewed Jan 15, 2016
View reviewed changes

domenic mentioned this pull request Jan 15, 2016

Implement Element.insertAdjacentHTML() #1355

Closed

Sebmaster force-pushed the feature/new-parse5 branch 2 times, most recently from 00c0f4b to d1aa072 Compare January 17, 2016 17:05

Sebmaster mentioned this pull request Jan 26, 2016

Issue with tagName case-sensitivity when creating SVG content. #620

Closed

Sebmaster force-pushed the feature/new-parse5 branch from d1aa072 to 7d4c68a Compare January 26, 2016 13:13

Sebmaster force-pushed the feature/new-parse5 branch 2 times, most recently from 2cbcd85 to 07a18f7 Compare February 8, 2016 12:53

Sebmaster mentioned this pull request Mar 1, 2016

Don't interpret script tags when type is not in spec ECMA262 #1406

Closed

Fix wrong test execution order

f04e56c

During script execution, the document wasn't actually loaded yet, but our old parser implementation loaded the whole document before executing scripts.

Sebmaster and others added 5 commits March 5, 2016 02:53

Enable isTrusted to be set via privateData

1f7e173

Update stylesheets if their text content changes

9c5a483

(Nearly) completely implement script element

91beec0

Use new parse5 streaming interface

4703e66

start and end got renamed in parse5's location info

333ca27

Sebmaster force-pushed the feature/new-parse5 branch from 07a18f7 to 333ca27 Compare March 5, 2016 02:15

Sebmaster added 3 commits March 5, 2016 03:19

Fix template elements

7f18c07

Fix document types in document adapter

7b11160

Fix wrong script tests

2d71252

Sebmaster force-pushed the feature/new-parse5 branch from e8c5051 to 2d71252 Compare March 5, 2016 15:57

domenic mentioned this pull request Mar 10, 2016

Correct the logic as to uppercase if NOT xhtml AND html. #1415

Closed

Sebmaster added 2 commits March 25, 2016 12:07

Implement some more script loading semantics

9fa419e

Fix incorrect tests

3974157

MeirionHughes mentioned this pull request Aug 18, 2016

TypeError: Cannot read property 'line' of undefined aurelia/template-lint#92

Closed

papandreou mentioned this pull request Aug 19, 2016

Update jsdom to 9.4.2. assetgraph/assetgraph#592

Merged

domenic mentioned this pull request Nov 6, 2016

document.write cannot handle partial element writes #1648

Open

domenic force-pushed the master branch from 8908e70 to 2161116 Compare February 26, 2017 05:47

Zirro mentioned this pull request May 13, 2017

Upgrade parse5 to 3.0.2 #1849

Merged

domenic mentioned this pull request Jan 16, 2018

Overhaul HTML parsing infrastructure #2098

Merged

domenic mentioned this pull request Nov 1, 2018

Usage of external scripts doesn't block HTML parsing #2413

Open

domenic force-pushed the master branch from 7d3b123 to ae038bf Compare January 2, 2020 00:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use new parse5 streaming interface #1316

Use new parse5 streaming interface #1316

Sebmaster commented Dec 9, 2015

Sebmaster commented Dec 9, 2015

Sebmaster Dec 9, 2015

domenic commented Dec 9, 2015

Sebmaster commented Dec 9, 2015

Sebmaster commented Dec 12, 2015

sykaeh commented Jan 14, 2016

Sebmaster commented Jan 15, 2016

inikulin Jan 15, 2016

domenic commented Jan 15, 2016

Sebmaster commented Jan 26, 2016

MeirionHughes commented Aug 18, 2016

TimothyGu commented Jan 16, 2018

Use new parse5 streaming interface #1316

Are you sure you want to change the base?

Use new parse5 streaming interface #1316

Conversation

Sebmaster commented Dec 9, 2015

Sebmaster commented Dec 9, 2015

Sebmaster Dec 9, 2015

Choose a reason for hiding this comment

domenic commented Dec 9, 2015

Sebmaster commented Dec 9, 2015

Sebmaster commented Dec 12, 2015

sykaeh commented Jan 14, 2016

Sebmaster commented Jan 15, 2016

inikulin Jan 15, 2016

Choose a reason for hiding this comment

domenic commented Jan 15, 2016

Sebmaster commented Jan 26, 2016

MeirionHughes commented Aug 18, 2016

TimothyGu commented Jan 16, 2018