[WIP] 15270 Parameterize DOM Node Traversal Methods #15503

DominoTree · 2017-02-11T02:23:08Z

Think this is getting pretty close now.

Made some ugly edits to try and ensure exact functionality is preserved with iterators. Once I identify the issue I'm seeing, I'll clean them up.

I've also temporarily made all of the places calling iterators with the new type parameters only use Node or Element - this way they should have identical functionality as they did before, which should make debugging the iterators themselves easier.

Once the iterators are done, we can clean up where they're being called from as well.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #15270 (github issue number if applicable).

There are tests for these changes OR
These changes do not require tests because _____

This change is

highfive · 2017-02-11T02:23:13Z

Heads up! This PR modifies the following files:

@fitzgen: components/script/dom/document.rs, components/script/dom/nodeiterator.rs, components/script/dom/servoparser/mod.rs, components/script/dom/htmlcollection.rs, components/script/dom/node.rs, components/script/dom/documentfragment.rs, components/script/dom/htmlfieldsetelement.rs, components/script/dom/text.rs, components/script/dom/characterdata.rs, components/script/dom/servoparser/html.rs, components/script/dom/range.rs, components/script/dom/htmloptgroupelement.rs, components/script/dom/htmlselectelement.rs, components/script/dom/htmltableelement.rs, components/script/dom/htmlimageelement.rs, components/script/dom/element.rs, components/script/dom/htmltablecellelement.rs, components/script/dom/htmloptionelement.rs, components/script/lib.rs, components/script/dom/htmlbodyelement.rs, components/script/devtools.rs
@KiChjang: components/script/dom/document.rs, components/script/dom/nodeiterator.rs, components/script/dom/servoparser/mod.rs, components/script/dom/htmlcollection.rs, components/script/dom/node.rs, components/script/dom/documentfragment.rs, components/script/dom/htmlfieldsetelement.rs, components/script/dom/text.rs, components/script/dom/characterdata.rs, components/script/dom/servoparser/html.rs, components/script/dom/range.rs, components/script/dom/htmloptgroupelement.rs, components/script/dom/htmlselectelement.rs, components/script/dom/htmltableelement.rs, components/script/dom/htmlimageelement.rs, components/script/dom/element.rs, components/script/dom/htmltablecellelement.rs, components/script/dom/htmloptionelement.rs, components/script/lib.rs, components/script/dom/htmlbodyelement.rs, components/script/devtools.rs

highfive · 2017-02-11T02:23:14Z

Warning

These commits modify script code, but no tests are modified. Please consider adding a test!

KiChjang · 2017-02-16T21:45:10Z

r? @jdm

DominoTree · 2017-02-16T23:14:30Z

Status -

At least one of my iterators is currently broken and I'm trying to figure out which one(s) are incorrect. I've rewritten them several times but I'm not currently sure which one to focus on - I think I may have been staring at this too long and need a new set of eyes.

Once the iterators are working, I'd like to add while() loops to all of them, as in the first two examples from #14451 so the parameter can be used for filtering if necessary.

Finally, I'll take advantage of that where the iterators are currently being called from.

nox · 2017-02-27T15:51:34Z

Did you manage to progress on this, @DominoTree?

jdm · 2017-02-27T16:06:36Z

I meant to take a look at this last week but got swamped by other work. If I don't get a chance to do so today or tomorrow, I'll hand it off to someone else.

jdm · 2017-02-27T16:09:45Z

Actually @nox graciously offered to take this off my plate.
r? @nox

DominoTree · 2017-02-27T16:11:06Z

Haven't had a chance to dig into this much further yet, but any guidance would be appreciated. I should have some time this week.

The first commit was my first attempt, and the second was mostly cleaning things up to isolate issues a bit more. I think there were a few other minor changes I made to my local copy to fix some bugs that I haven't pushed up, but I'll have to take a look at it and see where things ended up.

nox · 2017-03-01T11:58:43Z

@bors-servo try

bors-servo · 2017-03-01T11:58:48Z

⌛ Trying commit da60bb3 with merge 810a139...


        Auto merge of #15503 - DominoTree:15270-parameterize-traversal, r=<try>

[WIP] 15270 Parameterize DOM Node Traversal Methods  Think this is getting pretty close now. Made some ugly edits to try and ensure exact functionality is preserved with iterators. Once I identify the issue I'm seeing, I'll clean them up. I've also temporarily made all of the places calling iterators with the new type parameters only use `Node` or `Element` - this way they should have identical functionality as they did before, which should make debugging the iterators themselves easier. Once the iterators are done, we can clean up where they're being called from as well. ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #15270 (github issue number if applicable).  - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____   --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/15503)

nox · 2017-03-01T11:59:23Z

@bors-servo try-

Never mind, didn't notice the second commit. Will experiment locally.

DominoTree · 2017-03-01T13:27:40Z

I believe all of the iterators should have while let loops instead of simple if let to allow them to act as filters. I tried it locally and it didn't cause any further issues, but the page rendering was still broken.

nox · 2017-03-01T14:38:46Z

@DominoTree What loop do you mean?

DominoTree · 2017-03-01T15:05:48Z

Sorry, in the first commit, if you look at the impls for SiblingIterator ReverseSiblingIterator and FollowingIterator, they use while loops to continue over subsequent elements if they fail to cast the first element they encounter. I believe all of the iterators should probably behave this way.

nox · 2017-03-01T15:24:15Z

@DominoTree Please remove the second commit, I don't want the simplifications that undo the improvements from the first in the tree, and I don't want to review it either. I think I found the issue with your refactoring and will soon submit the review.

DominoTree · 2017-03-01T15:32:38Z

@nox I will get that reverted shortly.

nox

A few logic errors in the iterators and many nits.

nox · 2017-03-01T15:34:32Z

components/script/dom/characterdata.rs

@@ -225,12 +225,12 @@ impl CharacterDataMethods for CharacterData {

    // https://dom.spec.whatwg.org/#dom-nondocumenttypechildnode-previouselementsibling
    fn GetPreviousElementSibling(&self) -> Option<Root<Element>> {
-        self.upcast::<Node>().preceding_siblings().filter_map(Root::downcast).next()
+        self.upcast::<Node>().preceding_siblings::<Element>().next()


No need for a type annotation here.

nox · 2017-03-01T15:34:32Z

components/script/dom/characterdata.rs

    }

    // https://dom.spec.whatwg.org/#dom-nondocumenttypechildnode-nextelementsibling
    fn GetNextElementSibling(&self) -> Option<Root<Element>> {
-        self.upcast::<Node>().following_siblings().filter_map(Root::downcast).next()
+        self.upcast::<Node>().following_siblings::<Element>().next()


No need for a type annotation here.

nox · 2017-03-01T15:34:32Z

components/script/dom/document.rs

@@ -482,7 +482,7 @@ impl Document {
    /// https://github.com/w3c/web-platform-tests/issues/2122
    pub fn refresh_base_element(&self) {
        let base = self.upcast::<Node>()
-                       .traverse_preorder()
+                       .traverse_preorder::<Node>()


self.upcast::<Node>().traverse_preorder::<HTMLBaseElement>()

nox · 2017-03-01T15:34:32Z

components/script/dom/document.rs

@@ -679,7 +679,7 @@ impl Document {
                .map_or(false, |attr| &**attr.value() == name)
        };
        let doc_node = self.upcast::<Node>();
-        doc_node.traverse_preorder()
+        doc_node.traverse_preorder::<Node>()


I think you can remove both the type annotation and the .filter_map(Root::downcast) part. If it doesn't compile, you can still do:

doc_node.traverse_preorder::<HTMLAnchorElement>()

nox · 2017-03-01T15:34:32Z

components/script/dom/document.rs

@@ -1809,8 +1809,7 @@ impl Document {
    /// Iterate over all iframes in the document.
    pub fn iter_iframes(&self) -> impl Iterator<Item=Root<HTMLIFrameElement>> {
        self.upcast::<Node>()
-            .traverse_preorder()
-            .filter_map(Root::downcast::<HTMLIFrameElement>)
+            .traverse_preorder::<HTMLIFrameElement>()


I'm not sure this type annotation is needed.

nox · 2017-03-01T15:34:32Z

components/script/dom/node.rs

-                self.current = Some(next_sibling);
-                return Some(current);
+                self.current = Some(next_sibling.clone());
+                return Root::downcast::<T>(current);


Same remark about None, the iterator should never return None until it's done with the whole tree.

nox · 2017-03-01T15:34:32Z

components/script/dom/node.rs

            }
            self.depth -= 1;
        }
        debug_assert!(self.depth == 0);
        self.current = None;
-        Some(current)
+        //TODO: dominotree - not sure how to handle failures here - is returning None okay if it can't cast to T?


Same remark about downcasting and unwrap.

nox · 2017-03-01T15:34:32Z

components/script/dom/node.rs

+            self.current = Some(first_child.clone());
+            if let Some(first_child) = Root::downcast::<T>(first_child) {
+                return Some(first_child);
+            }


That does not seem correct to me.

nox · 2017-03-01T15:34:32Z

components/script/dom/node.rs

-                            if self.child_elements().any(|c| c.upcast::<Node>() != child) {
-                                return Err(Error::HierarchyRequest);
+                            if self.children::<Element>().any(|c| c.upcast::<Node>() != child) {
+                                    return Err(Error::HierarchyRequest);


Nit: indentation changed.

nox · 2017-03-01T15:34:32Z

components/script/lib.rs

@@ -19,6 +19,8 @@

 #![deny(unsafe_code)]
 #![allow(non_snake_case)]
+// TODO: dominotree remove this
+#![allow(unrooted_must_root)]


DominoTree · 2017-03-01T16:49:07Z

@nox I hadn't reverted that commit yet and I just sat down to do it, should I go ahead and do it now? Or should I add your changes here that are applicable and then rebase?

nox · 2017-03-01T17:37:40Z

Yes, but I want it removed, not reverted as in what git revert does.

DominoTree · 2017-03-01T17:40:28Z

Okay, the commit is removed.

nox · 2017-03-01T18:08:51Z

Ok! Now all my comments appear on the Files tab, address them and I'll make another review pass. Thanks for your contribution and keeping up with it!

nox

The iterators are still broken and an additional nit.

nox · 2017-03-02T10:43:51Z

components/script_layout_interface/wrapper_traits.rs

@@ -91,7 +91,7 @@ pub trait LayoutNode: Debug + GetLayoutData + TNode {
            current: self.last_child(),
        })
    }
-
+    


Nit: broken indentation.

nox · 2017-03-02T10:43:51Z

components/script/dom/htmlcollection.rs

@@ -221,7 +221,7 @@ impl HTMLCollection {
    pub fn elements_iter_after<'a>(&'a self, after: &'a Node) -> impl Iterator<Item=Root<Element>> + 'a {
        // Iterate forwards from a node.
        HTMLCollectionElementsIter {
-            node_iter: after.following_nodes(&self.root),
+            node_iter: after.following_nodes::<Node>(&self.root),


This type annotation is useless, given node_iter must be an iterator that yields Root<Node> values.

nox · 2017-03-02T10:43:51Z

components/script/dom/htmlcollection.rs

@@ -235,7 +235,7 @@ impl HTMLCollection {
    pub fn elements_iter_before<'a>(&'a self, before: &'a Node) -> impl Iterator<Item=Root<Element>> + 'a {
        // Iterate backwards from a node.
        HTMLCollectionElementsIter {
-            node_iter: before.preceding_nodes(&self.root),
+            node_iter: before.preceding_nodes::<Node>(&self.root),


This type annotation is useless, given node_iter must be an iterator that yields Root<Node> values.

nox · 2017-03-02T10:43:51Z

components/script/dom/htmltableelement.rs

@@ -138,7 +138,7 @@ impl HTMLTableElement {
    fn get_rows(&self) -> TableRowFilter {
        TableRowFilter {
            sections: self.upcast::<Node>()
-                          .children()
+                          .children::<Node>()


You forgot to simplify this one.

nox · 2017-03-02T10:43:51Z

components/script/dom/node.rs

+            if let Some(first_child) = Root::downcast::<T>(first_child) {
+                self.current = Some(Root::upcast::<Node>(first_child.clone()));
+                return Some(first_child);
+            }


I don't understand what you are trying to do by repeating the loop, and current is still never yielded again.

DominoTree · 2017-03-02T16:44:54Z

@nox As far as the iterators go, I was working based on the proof-of-concept linked to in #15270 and the POC iterators seemed to work properly in some brief testing.

https://github.com/frewsxcv/servo/blob/04a3efd4f2e2d843c72b55fc6767b15c7bd3e26a/components/script/dom/node.rs#L1136

Would a better approach be to simply attempt to upcast/downcast the values which would be returned from the existing iterators as they exist in the master branch? If so, how do we handle cases where we encounter elements which cannot be cast to the desired type? Do we loop until we find an element which can be successfully cast to that type? Do we act as if there are no more elements to iterate over?

This is really the part where I'm having trouble figuring out what the correct approach is.

DominoTree · 2017-03-24T12:45:17Z

@nox updated those methods, and it's still exhibiting the same behavior. I'll try to dig into it some more today - let me know if anything jumps out at you.

nox

I think I found the issue in your current code.

nox · 2017-04-04T11:46:09Z

components/script/dom/node.rs

+    pub fn inclusively_following_siblings<T>(&self) -> SiblingIterator<T>
+        where T: DerivedFrom<Node>
+    {
+        SiblingIterator::<T> {


Nit: the type annotation isn't required here.

nox · 2017-04-04T11:46:09Z

components/script/dom/node.rs

+        where T: DerivedFrom<Node>
+    {
+        SiblingIterator::<T> {
+            current: Root::downcast::<T>(Root::from_ref(self)),


This is incorrect, the node following self may be a T, but if self is not a T, current will be set to None.

You probably want:

SiblingIterator { current: Root::downcast(Root::from_ref(self)).or_else(|| get_next_sibling(self)), phantom: PhantomData, }

nox · 2017-04-04T11:46:09Z

components/script/dom/node.rs

+        where T: DerivedFrom<Node>
+    {
+        ReverseSiblingIterator::<T> {
+            current: Root::downcast::<T>(Root::from_ref(self)),


Same remark here:

Root::downcast(Root::from_ref(self)).or_else(|| get_previous_sibling(self))

DominoTree · 2017-04-05T01:24:26Z

@nox Thanks, that makes sense - I made those changes and was able to debug a bit more.

On servo.org, when it's inserting the element for <h1 id='page-title' class='page-header'>, the call to self.GetDocumentElement() at document.rs:616 returns None

The code in that method is simply self.upcast::<Node>().children().next() - so it would make sense that it would return None as that h1 element has no children, but I'm confused as to how this is supposed to work. If we're getting the document element, shouldn't it be traversing the DOM upwards?

DEBUG:html5ever::tree_builder: processing TagToken(Tag { kind: StartTag, name: Atom(\'h1\' type=static), self_closing: false, attrs: [Attribute { name: QualName { ns: Atom(\'\' type=static), local: Atom(\'id\' type=static) }, value: Tendril<UTF8>(owned: \"page-title\") }, Attribute { name: Qual
Name { ns: Atom(\'\' type=static), local: Atom(\'class\' type=static) }, value: Tendril<UTF8>(owned: \"page-header\") }] }) in insertion mode InBody
DEBUG:script::dom::document: Adding named element to document 0x7f14daaa9600: 0x7f14dab00480 id=page-title
WARN:servo: Panic hook called.
DEBUG:constellation::constellation: Sending log entry Warn("Panic hook called.").
The element is in the document, so there must be a document element. (thread ScriptThread PipelineId { namespace_id: PipelineNamespaceId(0), index: PipelineIndex(1) }, at /checkout/src/libcore/option.rs:785)
ERROR:servo: The element is in the document, so there must be a document element.
DEBUG:constellation::constellation: Received log entry Warn("Panic hook called.").

jdm · 2017-04-05T01:59:11Z

The document element should be the first child of the Document. self is the document, not the h1 element.

DominoTree · 2017-04-05T02:09:44Z

I suppose I could've scrolled up until I got to impl DocumentMethods for Document { 😃

bors-servo · 2017-04-05T14:28:50Z

☔ The latest upstream changes (presumably #15904) made this pull request unmergeable. Please resolve the merge conflicts.


        Parameterize node traversal methods

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

+where T: DerivedFrom<Node> {
+    let mut current = Root::from_ref(start);
+    //Should while loop here should probably be an if block
+    //or should we go through all the parents?


nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

+        where T: DerivedFrom<Node>
+    {
+        SiblingIterator {
+            current: Root::downcast(Root::from_ref(self)).or_else(||get_next_sibling(self)),


Nit: space after ||.

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

        ReverseSiblingIterator {
-            current: Some(Root::from_ref(self)),
+            current: Root::downcast(Root::from_ref(self)).or_else(||get_previous_sibling(self)),


Nit: space after ||.

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

-            current: Some(Root::from_ref(self)),
+    pub fn following_nodes<T>(&self, root: &Node) -> FollowingIterator<T> where T: DerivedFrom<Node> {
+        FollowingIterator::<T> {
+            current: Root::downcast::<T>(Root::from_ref(self)),


This isn't correct.

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

-            current: Some(Root::from_ref(self)),
+    pub fn preceding_nodes<T>(&self, root: &Node) -> PrecedingIterator<T> where T: DerivedFrom<Node> {
+        PrecedingIterator::<T> {
+            current: Root::downcast::<T>(Root::from_ref(self)),


This isn't correct.

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

@@ -729,15 +743,17 @@ impl Node {
        Ok(NodeList::new_simple_list(&window, iter))
    }

-    pub fn ancestors(&self) -> AncestorIterator {
+    pub fn ancestors(&self) -> AncestorIterator<Node> {


Why isn't this one parameterised?

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

        }
    }

-    pub fn inclusive_ancestors(&self) -> AncestorIterator {
+    pub fn inclusive_ancestors(&self) -> AncestorIterator<Node> {


Why isn't this one parameterised?

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

        ReverseSiblingIterator {
-            current: self.GetLastChild(),
+            current: Root::downcast(Root::from_ref(self)).or_else(||get_last_child(self)),


Nit: space after ||.

nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

+fn get_first_child<T>(start: &Node) -> Option<Root<T>>
+where T: DerivedFrom<Node> {
+    let mut current = Root::from_ref(start);
+    //The while loop here should probably just be an if block


nox · 2017-04-16T11:12:17Z

components/script/dom/node.rs

+fn get_last_child<T>(start: &Node) -> Option<Root<T>>
+where T: DerivedFrom<Node> {
+    let mut current = Root::from_ref(start);
+    //The while loop here should probably just be an if block


bors-servo · 2017-04-19T08:07:58Z

☔ The latest upstream changes (presumably #16521) made this pull request unmergeable. Please resolve the merge conflicts.

KiChjang · 2017-06-09T02:25:12Z

Is this PR wanted anymore?

DominoTree · 2017-06-09T04:37:30Z

@KiChjang Not sure what to do at this point - when I opened this PR, there were some DOM parsing issues that had been introduced by my changes. After fixing nits and trying a few different approaches (as well as several rounds of updates to make things merge cleanly), the same DOM parsing issues still exist, and I have been unable to figure out how to resolve them.

I'd be glad to continue putting effort into this PR, but until the fundamental DOM issues can be resolved, there doesn't seem to be much value here, and I'm really not sure how to move forward.

highfive assigned KiChjang Feb 11, 2017

highfive added the S-awaiting-review label Feb 11, 2017

highfive assigned jdm and unassigned KiChjang Feb 16, 2017

highfive assigned nox and unassigned jdm Feb 27, 2017

nox requested changes Mar 1, 2017

View changes

nox requested changes Mar 2, 2017

View changes

nox added S-needs-code-changes and removed S-awaiting-review labels Mar 24, 2017

highfive added S-awaiting-review and removed S-needs-code-changes labels Mar 24, 2017

nox requested changes Apr 4, 2017

View changes

nox added S-needs-code-changes and removed S-awaiting-review labels Apr 4, 2017

highfive added S-awaiting-review and removed S-needs-code-changes labels Apr 5, 2017

highfive added the S-needs-rebase label Apr 5, 2017

nox added S-needs-code-changes and removed S-awaiting-review labels Apr 6, 2017

Parameterize node traversal methods

Loading status checks…

bf6d68b

highfive added S-awaiting-review and removed S-needs-code-changes labels Apr 16, 2017

nox removed the S-needs-rebase label Apr 16, 2017

nox requested changes Apr 16, 2017

View changes

nox added S-needs-code-changes and removed S-awaiting-review labels Apr 16, 2017

highfive added the S-needs-rebase label Apr 19, 2017

DominoTree closed this Jul 22, 2017

tamird mentioned this pull request Aug 19, 2017

Parameterize node traversal helper methods over desired return type #15270

Open

servo / servo

Join GitHub today

[WIP] 15270 Parameterize DOM Node Traversal Methods #15503

[WIP] 15270 Parameterize DOM Node Traversal Methods #15503

Conversation

DominoTree commented Feb 11, 2017 • edited

highfive commented Feb 11, 2017

highfive commented Feb 11, 2017

KiChjang commented Feb 16, 2017

DominoTree commented Feb 16, 2017

nox commented Feb 27, 2017

jdm commented Feb 27, 2017

jdm commented Feb 27, 2017

DominoTree commented Feb 27, 2017 • edited

nox commented Mar 1, 2017

bors-servo commented Mar 1, 2017

nox commented Mar 1, 2017

DominoTree commented Mar 1, 2017

nox commented Mar 1, 2017 • edited

DominoTree commented Mar 1, 2017

nox commented Mar 1, 2017

DominoTree commented Mar 1, 2017

nox left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

DominoTree commented Mar 1, 2017 • edited

nox commented Mar 1, 2017

DominoTree commented Mar 1, 2017

nox commented Mar 1, 2017

nox left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

DominoTree commented Mar 2, 2017

DominoTree commented Mar 24, 2017

nox left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

DominoTree commented Apr 5, 2017 • edited

jdm commented Apr 5, 2017

DominoTree commented Apr 5, 2017

bors-servo commented Apr 5, 2017

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bors-servo commented Apr 19, 2017

KiChjang commented Jun 9, 2017

DominoTree commented Jun 9, 2017

DominoTree commented Feb 11, 2017 •

edited

DominoTree commented Feb 27, 2017 •

edited

nox commented Mar 1, 2017 •

edited

DominoTree commented Mar 1, 2017 •

edited

DominoTree commented Apr 5, 2017 •

edited