HTML parser test failures after #6993

[TRowbotham]

I'm seeing 2 test failures with the recent change from #6399; in particular, the change to use the "in body insertion mode" directly, instead of reprocessing with the current insertion mode. In both cases[1][2], we lose out on foster parenting with the switch to using the "in body insertion mode", which causes the <p>baz part to be inserted into the table rather than outside the table. Switching back to previous behavior where reprocessing the token uses the current insertion mode seems to resolve the failures.

[1] https://github.com/html5lib/html5lib-tests/blob/master/tree-construction/tests9.dat#L270
[2] https://github.com/html5lib/html5lib-tests/blob/master/tree-construction/tests10.dat#L236

[annevk]

cc @whatwg/html-parser

[zcorpan]

Thanks @TRowbotham!

The rationale for changing the reprocess the token part was this:

However, the spec doesn't say to change insertion mode, or use the rules for "in body", only to reprocess the token. Won't this just be reprocessed in "in foreign content" again, at least if nothing was popped? Would things work if it said "Process the token using the rules for the "in body" insertion mode."?

#5117 (comment)

So, evidently, things wouldn't work with "process the token using the rules for". What about my original question?

Given this case:

<svg id=svg></svg>
<script>
svg.innerHTML = '<p>';
</script>

What happens if the spec says to reprocess the token? Does it reprocess the token still in the "in foreign content" mode, and again say to reprocess the token (i.e. get stuck in an infinite loop)?

If yes, maybe it can be fixed by setting a flag before reprocessing the token, and checking for that flag in the tree construction dispatcher (and resetting the flag there also)?

[TRowbotham]

Yes, the given case causes the parser to get stuck in an infinite loop if it just says reprocess the token.

It seems I slightly misunderstood the meaning of reprocess the token, but perhaps it works in our favor. I understood it as "reprocess the token using the current insertion mode", and this seemingly appears to yield the desired result; the given case does not result in an infinite loop and the 2 test failures I was seeing go away.

Is there anything wrong with changing the spec to read "reprocess the token using the current insertion mode" instead of "Process the token using the rules for the "in body" insertion mode."? If the intent is to break out of foreign content to whatever the last non-foreign element was, then using whatever the last insertion mode was kinda makes sense.

[zcorpan]

I understood it as "reprocess the token using the current insertion mode"

I can't confidently say whether or not this is the right interpretation. This should probably be made clearer. 🙂

Is there anything wrong with changing the spec to read "reprocess the token using the current insertion mode" instead of "Process the token using the rules for the "in body" insertion mode."?

As long as it does what we need and is unambiguous, it's good. I'd like to hear what others think, in particular @hsivonen

[zcorpan]

Is there any difference between:

+    <p>Reprocess the token according to the rules given in the section corresponding to the current
+    <span>insertion mode</span> in HTML content.</p>

and

+    <p>Process the token according to the rules given in the section corresponding to the current
+    <span>insertion mode</span> in HTML content.</p>

(i.e., "Reprocess the token" vs "Process the token")

The clause in "Any other end tag" in foreign content says "Process the token ...".

[zcorpan]

PR: #6455

[hsivonen]

This is the sort of thing that's hard to review without writing the code. I'll try to get to the code soon.

[TRowbotham]

PR #6455 also appears to resolve #1376