Infinite loop while Building

[dginev]

I have extracted a minimal example from the infinite loop reported at arxiv-vanity/engrafo#280

which converted 1404.2371 from arXiv. The minimal example is:

\documentclass{article}
\usepackage{amsmath,amssymb,amsthm}

\def\qed{\framebox(6,6){}}

\begin{document}

\begin{proof}
proof here
\end{proof}

\end{document}

which produces the log:

$ latexml nthroot.tex

latexml (LaTeXML version 0.8.2; revision a23e6ac2)
processing started Sun Feb 25 11:54:07 2018

(Digesting TeX nthroot...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/TeX.pool.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/eTeX.pool.ltxml... 0.01 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/pdfTeX.pool.ltxml... 0.01 sec) 0.14 sec)
(Processing content /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/LaTeX.pool.ltxml... 0.16 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/article.cls.ltxml... 0.02 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsmath.sty.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsbsy.sty.ltxml... 0.00 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amstext.sty.ltxml... 0.00 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsopn.sty.ltxml... 0.00 sec) 0.06 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amssymb.sty.ltxml...
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsfonts.sty.ltxml... 0.00 sec) 0.03 sec)
(Loading /home/dreamweaver/perl5/lib/perl5/LaTeXML/Package/amsthm.sty.ltxml... 0.00 sec) 0.33 sec) 0.48 sec)
(Building...
(Loading compiled schema /home/dreamweaver/perl5/lib/perl5/LaTeXML/resources/RelaxNG/LaTeXML.model... 0.02 sec)...
Warning:perl:warn Deep recursion on subroutine "LaTeXML::Core::Document::find_insertion_point"
	at /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex; line 9 col 1
	at /home/dreamweaver/perl5/lib/perl5/LaTeXML/Core/Document.pm line 722
	In Core::Document[@0x56144c85bc38] at /home/dreamweaver/testbed/infinite_loop_latexml_post/nthroot.tex; line 9 col 1

Warning:perl:warn Deep recursion on subroutine "LaTeXML::Core::Document::find_insertion_point"
	at Anonymous String; line 0 col 0
	at /home/dreamweaver/perl5/lib/perl5/LaTeXML/Core/Document.pm line 897
	In Core::Document[@0x56144c85bc38] at Anonymous String; line 0 col 0

................................................................................

and keeps Building ad infinitum.

The framebox argument should be playing a major role here. pdflatex succeeds and creates the expected:

I am hunting for two separate fixes here:

1. Try to improve the Document building robustness to avoid this loop with the current master, so that we avoid needless timeouts / memory hogging jobs - the arXiv example allocated 5 GB of RAM in 16 minutes for me. (before i interrupted it)
2. Fix the actual infinite loop in this example, which should be a much more specific patch to framebox

[dginev]

This looks to be an interesting malfunction by find_insertion_point. It looks like:

• a ltx:rect needs to be deposited for the framebox
• it needs an intermediate ltx:text to be opened in order to contain it
• However, the first action taken in that method closes any currently open text node:

  $self->closeText_internal;    # Close any current text node.
  

• Hence, loop infinitely opening and closing text nodes, and creating actual empty ltx:text node in the libxml tree

[dginev]

Debugging some more, the current master claims that:

• ltx:p can contain ltx:rect with a ltx:text intermediary
• a direct canContain call however, returns false for ltx:text can contain ltx:rect

So there may also be a schema tweak needed. I will focus on ensuring schema mismatches don't lead to infinite loops in general until @brucemiller imparts schematic wisdom here, as a general case guard.

[brucemiller]

Problem with naming closeText_internal just closes a "text" node (ie. PCDATA), not an ltx:text element.

What's bizarre is that it would need to open ltx:picture to place a ltx:rect, but picture has autoOpen=>0.5 ! Presumably that means it's not preferred? Who wrote this stuff! I guess what's happening is that canContainIndirect says yes, but canContain says no!?!?!

[dginev]

Ah! That naming distinction "threw me in for a loop". So ltx:text remains open and the two fighting methods are the autoclose vs autoopen, got it.

I think I can add a guard that terminates early when a new auto-open attempts a name that has already been auto-closed (during a find_insertion_point recursive chain). Writing that PR now, will leave the mysterious autoOpen=>0.5 probabilistic magic to it creator :>

[brucemiller]

Probably just need more threads...

Not clear what's happening (computeIndirectModel is rather opaque, atm). Apparently canContainIndirect says ltx:rect CAN be inserted into ltx:p after opening ltx:text and ltx:picture; but says ltx:rect can NOT be inserted into ltx:text (not after opening ltx:picture)?

[dginev]

Adding new thread where we can discuss the generic guard - now implemented at #952

Onto the specific framebox schema mystery in this issue. I can confirm your observations. I wish I understood the ltx:rect element a bit better in order to comment, but it is certainly an "inline" use in the source, in order to draw the QED rectangle, so depositing it inside a text element sounds reasonable at a glance.

[dginev]

Also, it feels like we should add a schema-oriented test, using this \qed in text and math mode, to stay certain we have this nailed down going forward.

[brucemiller]

Actually was a bit fishier: computeIndirectModel decided that you could open a ltx:text when trying to insert ltx:rect into ltx:text (or the other way 'round); in any case, it created potential cycles. Thanks for narrowing down the test case; should be fixed now!