beamer: \column in a [fragile] frame swallows the document (a minipage whose \end is stored for later)

[tannevaled]

Six lines, real beamer.cls, and the document renders nothing:

\documentclass{beamer}
\begin{document}
\begin{frame}[fragile]{A}
\begin{columns}
\column{0.5\textwidth}
gauche
\end{columns}
\end{frame}
\begin{frame}{B}MARQUE\end{frame}
\end{document}

pages=0. Drop the [fragile] and it is 2 pages with everything on them.

Reach

It is what stands between the engine and three of the eighty talks that now have a tectonic PDF beside them — each renders 1 page against 7, 5 and 9. columns in a fragile frame is ordinary beamer.

What is known

• The verbatim capture is correct. texput.vrb receives exactly

  \frametitle{A}
  \begin{columns}
  \column{0.5\textwidth}
  gauche
  \end{columns}
  

• Every reconstruction of the re-entry works: the command form \frame{…columns…}, the exact shape beamer rebuilds (\frame<*>[][{,fragile=false}]{\begingroup…\endgroup}), and the same body in a normal frame. Only the real [fragile] environment fails.

• The cause is beamer's \column, which opens an environment and stores its \end for later (beamerbaseframecomponents.sty):

  \newcommand<>\beamer@columncom[2][\beamer@colmode]{%
    \beamer@colclose%
    \def\beamer@colclose{\end{minipage}\hfill\end{actionenv}\ignorespaces}%
    \begin{actionenv}#3\begin{minipage}…}

  so \begin{minipage} is opened by \column and its \end{minipage} only runs at the NEXT \column or at \end{columns}.

• This engine COLLECTS a minipage's body up front. Instrumented, the collector starts inside the .vrb (niveau=1, bpos=54/91) and ends at niveau=0, bpos=186/186 — it runs off the end of that file and eats the whole document behind it.

• engine: a collected environment body that finds no \end puts back what it read #121 already restores the input when a collected body finds no \end, but only when the scan stayed on ONE input level; here it crosses one, because a fragile frame's body comes back as a file of its own.

• Stopping the scan at the end of the file it began in was tried: the collector then stops at bpos=138 instead of 186 — better, but the document still renders 0 pages, so the swallow is not the whole story.

Bar

The six-line document renders 2 pages with gauche and MARQUE, and the three talks (0029f300…, 023ae352…, 042e4d92…) move from 1 page toward their reference.

[tannevaled]

Tried the obvious fix and measured it — it is a regression, recording so nobody pays for it twice.

\endcolumns begins with \beamer@colclose, whose own body begins with \end{minipage}. So the rule "expand \end<n> when it leads to an \end" was extended from one level to two (endMacroLeadsToEnd). That fixes the six-line reproducer — 2 pages with gauche and MARQUE, whole suite green — and over the corpora:

	pages	glyph paths	up	down
beamer	1197 → 1169 (−28)	202 558 → 198 483 (−4 075)	0	4
arXiv	4253 → 4253	unchanged	0	0

One talk goes 17 → 1 page, another 623 → 0 glyphs. Against the 80 talks with a tectonic PDF, the absolute page error goes 70 → 71.

Expanding an environment's end code two levels deep is too eager: it fires on environments whose \end<n> merely starts with a macro that starts with \end, and runs their terminator early. Reverted.

So the shape of the fix has to be last-resort rather than eager: the two-level expansion is only right at the point where the scan would otherwise run off the end of the file it began in — which is exactly where #121's restore currently gives up because the scan crossed a level. Stopping the scan at that boundary was also tried (it stops earlier, bpos=138 instead of 186) and is not sufficient on its own.

Both halves are measured now; what is missing is combining them: at the file boundary, expand what the pending \end{other} leads to instead of giving up.

[tannevaled]

Third approach measured, and it fails for a reason worth recording: a failed scan cannot be re-run.

The idea was to keep the narrow one-level rule as the first pass and use the two-level rule only as a LAST RESORT — restore the mouth exactly (buffer, position, line table, pending lists AND the stack of input levels: an ordinary inputMark cannot be restored across a file boundary, so markInputFull/restoreInputFull were added) and scan again allowing endMacroLeadsToEnd.

It does not work, and the first test to fail says why: \begin{multicols}{2}\end{multicols}.

\scanOptBracketToks peeks for the optional […] with expansion, so \end{multicols} is already expanded before the collector starts — into \csname endmulticols\endcsname and \gotex@endenv{multicols}, thirteen tokens with no \end among them. The collector therefore never matches, both passes run, and the second pass executes \gotex@endenv a second time: one group too many closed, and the document errors with Extra }, or forgotten \endgroup.

That is general, not specific to multicols. A scan that fails has already:

• closed groups (\gotex@endenv on nested \end{…}),
• expanded macros (expandsToEnd, runsCsname),
• and run whatever those expansions did.

Restoring the INPUT cannot undo any of it. So the retry shape is out.

What is left, and what I would try next: decide within the single pass, at the moment the scan is about to run off the end of the file it began in — at that point, and only there, consult endMacroLeadsToEnd on the \end{other} most recently stored in the body, rewind the body to it, and expand it in place. No second execution, no undo.

Recorded so the next pass starts from three measured dead ends rather than from scratch:

1. two-level rule eagerly — fixes the reproducer, costs 28 pages / 4 talks, tectonic error 70 → 71;
2. stopping the scan at the file boundary — stops earlier, still 0 pages;
3. restore-and-retry — unsound, a failed scan has already acted.

[tannevaled]

Closed by #176.

#129 caught only the case where the frame's own file ends at the swallow. The general one is that a column's \end{minipage} is deferred into \beamer@colclose (beamerbaseframecomponents.sty:281-283) and arrives through the next \column — which takes arguments, so the narrow expandsToEnd rule never saw it — or through \end{columns}, which was stored as somebody else's \end.

Both are now read as the \end they carry. The reduced case from this issue renders its three pages, a corpus talk goes 1 → 5 (tectonic 7) and another 2 → 4, exactly tectonic's; beamer's page error against the 80 reference renders falls 48 → 42.