Infinite loop while Building

[dginev]

I have extracted a minimal example from the infinite loop reported at arxiv-vanity/engrafo#280

which converted 1404.2371 from arXiv. The minimal example is:

\documentclass{article}
\usepackage{amsmath,amssymb,amsthm}

\def\qed{\framebox(6,6){}}

\begin{document}

\begin{proof}
proof here
\end{proof}

\end{document}

which produces the log:

$ latexml nthroot.tex

latexml (LaTeXML version 0.8.2; revision a23e6ac2)
processing started Sun Feb 25 11:54:07 2018

(Digesting TeX nthroot...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/TeX.pool.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/eTeX.pool.ltxml... 0.01 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/pdfTeX.pool.ltxml... 0.01 sec) 0.14 sec)
(Processing content /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/LaTeX.pool.ltxml... 0.16 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/article.cls.ltxml... 0.02 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsmath.sty.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsbsy.sty.ltxml... 0.00 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amstext.sty.ltxml... 0.00 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsopn.sty.ltxml... 0.00 sec) 0.06 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amssymb.sty.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsfonts.sty.ltxml... 0.00 sec) 0.03 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsthm.sty.ltxml... 0.00 sec) 0.33 sec) 0.48 sec)
(Building...
(Loading compiled schema /home/dreamweaver/perl5/lib/perl5/LaTeXML/resources/RelaxNG/LaTeXML.model... 0.02 sec)...
Warning:perl:warn Deep recursion on subroutine "LaTeXML::Core::Document::find_insertion_point"
	at /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex; line 9 col 1
	at /home/dreamweaver/perl5/lib/perl5/LaTeXML/Core/Document.pm line 722
	In Core::Document[@0x56144c85bc38] at /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex; line 9 col 1

Warning:perl:warn Deep recursion on subroutine "LaTeXML::Core::Document::find_insertion_point"
	at Anonymous String; line 0 col 0
	at /home/dreamweaver/perl5/lib/perl5/LaTeXML/Core/Document.pm line 897
	In Core::Document[@0x56144c85bc38] at Anonymous String; line 0 col 0

................................................................................

and keeps Building ad infinitum.

The framebox argument should be playing a major role here. pdflatex succeeds and creates the expected:

I am hunting for two separate fixes here:

1. Try to improve the Document building robustness to avoid this loop with the current master, so that we avoid needless timeouts / memory hogging jobs - the arXiv example allocated 5 GB of RAM in 16 minutes for me. (before i interrupted it)
2. Fix the actual infinite loop in this example, which should be a much more specific patch to framebox

[dginev]

This looks to be an interesting malfunction by find_insertion_point. It looks like:

• a ltx:rect needs to be deposited for the framebox
• it needs an intermediate ltx:text to be opened in order to contain it
• However, the first action taken in that method closes any currently open text node:

  $self->closeText_internal;    # Close any current text node.
  

• Hence, loop infinitely opening and closing text nodes, and creating actual empty ltx:text node in the libxml tree

[dginev]

Debugging some more, the current master claims that:

• ltx:p can contain ltx:rect with a ltx:text intermediary
• a direct canContain call however, returns false for ltx:text can contain ltx:rect

So there may also be a schema tweak needed. I will focus on ensuring schema mismatches don't lead to infinite loops in general until @brucemiller imparts schematic wisdom here, as a general case guard.

[brucemiller]

Problem with naming closeText_internal just closes a "text" node (ie. PCDATA), not an ltx:text element.

What's bizarre is that it would need to open ltx:picture to place a ltx:rect, but picture has autoOpen=>0.5 ! Presumably that means it's not preferred? Who wrote this stuff! I guess what's happening is that canContainIndirect says yes, but canContain says no!?!?!

[dginev]

Ah! That naming distinction "threw me in for a loop". So ltx:text remains open and the two fighting methods are the autoclose vs autoopen, got it.

I think I can add a guard that terminates early when a new auto-open attempts a name that has already been auto-closed (during a find_insertion_point recursive chain). Writing that PR now, will leave the mysterious autoOpen=>0.5 probabilistic magic to it creator :>

[brucemiller]

Probably just need more threads...

Not clear what's happening (computeIndirectModel is rather opaque, atm). Apparently canContainIndirect says ltx:rect CAN be inserted into ltx:p after opening ltx:text and ltx:picture; but says ltx:rect can NOT be inserted into ltx:text (not after opening ltx:picture)?

[dginev]

Adding new thread where we can discuss the generic guard - now implemented at #952

Onto the specific framebox schema mystery in this issue. I can confirm your observations. I wish I understood the ltx:rect element a bit better in order to comment, but it is certainly an "inline" use in the source, in order to draw the QED rectangle, so depositing it inside a text element sounds reasonable at a glance.

[dginev]

Also, it feels like we should add a schema-oriented test, using this \qed in text and math mode, to stay certain we have this nailed down going forward.

[brucemiller]

Actually was a bit fishier: computeIndirectModel decided that you could open a ltx:text when trying to insert ltx:rect into ltx:text (or the other way 'round); in any case, it created potential cycles. Thanks for narrowing down the test case; should be fixed now!

[dginev]

Sweet!

[dginev]

Deyan from the future here. I've started wondering whether we shouldn't define \qed at TeX.pool and lock it for good, especially if we could give it some "semantics" in the final format to justify doing that.

And here is the reason I am recording this thought:

\newcommand{\qed}{{ \hfill
  {\unskip\kern 6pt\penalty 500
  \raise -2pt\hbox{\vrule\vbox to 6pt{\hrule width 6pt
  \vfill\hrule}\vrule} \par}   }}

latexml doesn't really draw this right, but of course - even if it did - the resulting HTML is horrifying to inspect by humans, just as the original TeX is. So, alternatively, maybe we should get enough upgrades in to draw the box correctly? :> Just a thought for now.