tikz: the picture environment starts, and a class loads its size file - #3
Merged
Conversation
\begin{tikzpicture} now runs and \draw draws: TikZ lines, rectangles and
circles reach the page at the right coordinates, with their colour and
thickness. Eight more places where the engine answered differently from TeX,
each found by running the real sources and, where TeX's own answer was in
question, measured against a real TeX first.
\active — plain TeX's name for category 13 — was missing. A package makes a
character active by name (\catcode`\;=\active), and tikz does so a third of
the way through its file; the block failed and everything defined after it
was lost with it, which is why \draw was never installed.
A file name is expanded as it is read, as TeX's file-name scanner does. pgf
loads each of its modules with \input{pgfmodule\pgf@temp.code.tex}, so a
literal reading found no file and the module silently never loaded — that is
where \pgfnodeparttextbox and the whole shapes module went. The lines of an
\input file are now discounted from the document's own numbering too, as a
class's already were, so an error still points at the right line.
A name \csname brings into existence means \relax, and that definition is
LOCAL. A package asks whether a control sequence exists by expanding \csname
inside a group it then closes, so the question leaves no trace; defining it
globally made every later test answer "yes". pgf asks that way whether it is
running under LuaTeX — and then emitted its Lua branch, hundreds of lines of
Lua onto the page.
\lccode/\uccode are real tables that \lowercase/\uppercase map through, which
is how a package substitutes one character for another (pgfmath builds its
parser's catcode block that way). \colorlet takes xcolor's optional model
argument, which pgf passes whenever the source colour is a named one, and the
standard colour names are published in readable form so \draw[red] is
recognised as a colour at all. \setlength assigns a register reference such
as {\skip\footins}, and \@settopoint rounds a length to whole points.
Together these leave the page clean: loading tikz used to deposit 620 KB of
package internals on it, and now deposits nothing.
Arithmetic finally works on the dimension parameters the engine models as
primitives of its own rather than as registers. \advance\hsize did nothing
and \divide\hsize\p@ produced nonsense — which only surfaced once the class
really loaded its size option file (the file name is built from a macro, so
it had never resolved before): the computed text width came out as one
point, truncating every paragraph. A document now gets the geometry its
class computed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Aug 26, 2026
\newcommand\foo[\somecount]{…} defined a command of ZERO arguments. The reader
walked the bracket's tokens, kept the digits and skipped everything else, so a
count that arrived as a register — or as \the<register> — read as nothing.
That is not a missing feature, it is silent damage: the command then consumed
none of the arguments its caller passed, and each one was left in the input and
TYPESET. beamer generates its whole overlay layer this way,
\newcommand\csname beamerx@…\endcsname[\beamer@argscount]{#3}
so every \defbeamertemplate printed its template body onto the page instead of
storing it. An empty beamer document came out 5 pages of that debris; it is now
2. The same shift also mis-bound the arguments further down the chain — the
overlay specification \beamer@sortzero passes as {} was picked up as the first
real argument, so \defbeamertemplate{tst}{fam}{…} built its key as
\beamer@@tmpop@@tst instead of \beamer@@tmpop@tst@fam.
A bracket holding a control sequence now goes to the engine's own integer
scanner, on an isolated input stack so nothing it leaves unread escapes into the
document. A bracket of plain digits keeps the old path.
One test expectation moves. It asserted that \newcommand\c[\relax 1]{…} takes
ONE argument — digits kept, control sequences ignored. Real LaTeX does not do
that: it scans a number, raises "Missing number, treated as zero", and defines a
command of no arguments. Checked with tectonic, which also confirms the register
form: \newcount\nn \nn=1 \newcommand\ca[\nn]{<#1>} makes \ca{X} give <X>.
tannevaled
added a commit
that referenced
this pull request
Aug 27, 2026
…mes expand (#99) Two defects that turn out to be one story: no beamer theme had ever been loaded, and every keyval value was silently lost. 1. \@ifnextchar does not INSERT the branch it picks — it stores it first. ltdefns.dtx: \long\def\@ifnextchar#1#2#3{\let\reserved@d=#1 \def\reserved@a{#2}\def\reserved@b{#3}\futurelet\@let@token\@ifnch} and \def\reserved@b{#3} SCANS the branch as a macro body, which halves ## a second time (tex.web §479: a # followed by a # stores one #). This engine's \@ifnextchar is a primitive that inserted the branch verbatim, skipping it. keyval is built on exactly that: \def\define@key#1#2{\@ifnextchar[{\KV@def{#1}{#2}}{\long\@namedef{KV@#1@#2}####1}} The #### is halved once into \define@key's own body and a second time by \@ifnextchar, leaving the one # that \def reads as the parameter text #1. Without it the generated macro's parameter text was ##1 and bound nothing: \setkeys{fam}{k=VAL} gave <<>>. Real LaTeX gives \long macro:#1-><<#1>> where this engine gave macro:##1-><<#1>>. \@ifstar and \@ifnextbracket pick their branch the same way and are halved for the same reason. 2. \usepackage, \documentclass and \LoadClass now EXPAND their {name}. Checked in one line: \def\nm{marker}\usepackage{zz\nm} loads zzmarker.sty in real LaTeX and loaded nothing here. beamer builds every theme file name that way — \beamer@calltheme does \usepackage[{#1}]{beamertheme\beamer@themename} — so \usetheme{default} asked for a package literally named "beamertheme\beamer@themename". Together: the frametitle template (beamerouterthemedefault.sty) finally exists, so a FRAME TITLE renders; and because key values no longer leak, the theme files stop contributing four pages of debris. On the minimal document that is exactly the reference — n frames make n pages, title and body both present, page 364.2x273.15. Measured over 200 real beamer talks: 0 content lost, pages 1061 -> 1095, vector paths 155106 -> 174863. Eight documents stop producing a sized page and each was checked by hand: four are corpus FRAGMENTS with no \begin{document} (0 pages is what real LaTeX gives them), one is a talk with no frame at all, one is a fragile frame whose verbatim body this engine cannot render either way, one is unchanged when re-run cold, and one (039a234b02c5…) trips the runaway guard on main TOO — the guard now fires sooner, so 40KB of debris becomes none. Over 200 arXiv papers: the single apparent loss renders 2.46MB in both binaries when re-run cold; it had exceeded the sweep's time cap.
tannevaled
added a commit
that referenced
this pull request
Aug 30, 2026
…#129) \end<name> can reach the terminator a body scanner is hunting through ONE more parameterless macro, not only as its first token. beamer's columns are built that way (beamerbaseframecomponents.sty): \newcommand<>\beamer@columncom[2][\beamer@colmode]{% \beamer@colclose \def\beamer@colclose{\end{minipage}\hfill\end{actionenv}\ignorespaces}% \begin{actionenv}#3\begin{minipage}…} so \column opens a minipage and STORES its \end{minipage} in \beamer@colclose, to run at the next \column or at \end{columns}. \endcolumns therefore begins with \beamer@colclose, not with \end, and the scanner stores the \end{columns} raw and carries on — off the end of the file it is reading and into the document behind it. Six lines rendered ZERO pages (issue #127): \begin{frame}[fragile]{A} \begin{columns} \column{0.5\textwidth} gauche \end{columns} \end{frame} The scan now remembers where it stored its last \end{other}. When it is about to leave the file it began in, and that \end leads somewhere after all, it rewinds the body to that point, puts the stored tail back, and expands \end<other> in place. Nothing is re-EXECUTED: what goes back was gathered, never run. That distinction is the whole design, and two other shapes were measured first: - Applying the deeper rule EAGERLY fixes the same case and costs 28 pages over 200 talks, one of them from 17 pages to 1; the error against 80 tectonic PDFs rises from 70 to 71. It runs environments' terminators early. - Restoring the input and re-running the whole scan is unsound: a failed scan has already closed groups (\gotex@endenv) and expanded macros, and putting the INPUT back undoes none of it. \begin{multicols}{2}\end{multicols} shows it — \scanOptBracketToks peeks for the optional [ WITH expansion, so the \end is already \csname endmulticols\endcsname + \gotex@endenv{multicols} before the collector starts, and a second pass closes one group too many. Measured over 200 talks and 200 arXiv papers: not one document changes — 1197 pages and 202 558 glyphs on one side, 4253 and 9 561 555 on the other, identical before and after, and the absolute page error against the 80 talks with a tectonic PDF stays at 70. It closes a hole rather than moving anything.
tannevaled
added a commit
that referenced
this pull request
Aug 31, 2026
\newcommand{\cmd}[n][default] makes the FIRST of the n parameters optional: \cmd is
\@Protected@testopt, which supplies {default} when no [ follows (latex.ltx:1187-1199,
\@xargdef), and the remaining n-1 are grabbed as usual. The token path already does
this (engine.go's optArg/optDefault); the maths source path read all n as MANDATORY.
That is worse than not expanding at all. With the optional argument written out the
macro still "matched", by taking the brackets themselves as arguments:
\qbin [p]{N}{k} -> \genfrac []{0pt}{}{p}{]}_{[}{N}{k}
\qbin {N}{k}+1 -> \genfrac []{0pt}{}{k}{+}_{N}1
so the formula came out silently wrong, the + of an expression becoming an argument.
Without it, nothing matched and the formula was dropped — which is how a paper's own
\newcommand{\qbinom}[3][q]{\genfrac[]{0pt}{}{#2}{#3}_{#1}} lost 12 equations.
Measured over 200 arXiv papers and 200 beamer talks: 5 documents change for +1234
glyphs. One repaginates, 362 -> 363 against tectonic's 333 — a document already 29
pages long of it — and the page error against the 157 renders moves 669 -> 670.
beamer does not move by one glyph.
tannevaled
added a commit
that referenced
this pull request
Aug 31, 2026
…lumns} (#176) beamer opens a column's minipage and defers its \end into \beamer@colclose (beamerbaseframecomponents.sty:281-283): \newcommand<>\beamer@columncom[2][\beamer@colmode]{% \beamer@colclose \def\beamer@colclose{\end{minipage}\hfill\end{actionenv}\ignorespaces}% \begin{actionenv}#3…\begin{minipage}…} so the \end this scan hunts arrives through the NEXT \column, or through \end{columns} (:237, \endcolumns begins with \beamer@colclose). A raw scan sees neither: \column takes arguments where the narrow expandsToEnd rule only reaches parameterless macros, and \end{columns} is stored as somebody else's \end. Read raw, the first column swallowed its sibling and everything after the frame. A talk with two columns in a [fragile] frame rendered ONE page of its seven; issue #127 found this and #129 caught only the case where the frame's own file ends there. Both are now read as what they are: at \column the body ends and \beamer@colclose is emptied, its \end{minipage} having just been honoured (which is what beamer itself does after running it, :269); at \end{columns} the \end<columns> body is pushed so \beamer@colclose, and then its \end{minipage}, surface for the depth bookkeeping. Measured over 200 beamer talks and 200 arXiv papers: 5 talks change, two of them moving CLOSER to tectonic and none further — one 1 -> 5 pages (tectonic 7, +2268 glyphs), one 2 -> 4, exactly tectonic's. beamer gains 2885 glyphs and its page error against the 80 tectonic renders falls 48 -> 42, with one more exact match. arXiv does not move by one glyph.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation:
\begin{tikzpicture}starts and\drawdraws. TikZ lines, rectangles and circles reach the page at the right coordinates, with their color and their thickness.Eight more divergences, measured before the fix
\active(plain TeX's name for category 13) — missing;active a third of the way into its file: the block failed and everything that followed was lost, hence\drawnever installed\inputfile name is expanded\input{pgfmodule\pgf@temp.code.tex}→ no module was loading (hence\pgfnodeparttextboxmissing)\inputare counted out of the document's numbering\csnamecreates its\relaxlocally\lccode/\uccode= real tables,\lowercasegoes through them\colorlet[named]{…}(xcolor's optional argument)[named]landed on the page\draw[red]was not recognized as a color\setlength{\skip\footins}{…}and\@settopointCleanliness result: loading tikz used to dump 620 KB of package internals onto the page; it now dumps none (8,570 bytes versus 8,564 without tikz).
And a deep bug revealed along the way
Arithmetic did not work on the dimension parameters that the engine models as primitives rather than registers:
\advance\hsizedid nothing and\divide\hsize\p@gave garbage. This only surfaced once the class actually loaded its size file (its name is built by a macro, so it had never been resolved): the computed text width came out at 1 pt, which truncated every paragraph. A document now gets the geometry its class computes (345 pt forarticle10 pt on letter paper).Verification
go vetandgofmtclean.scripts/fidelity.sh --check(gotex vs real LaTeX): OK — the latter caught the text-width regression, which is fixed.Remaining for milestone (c):
\noderuns without error but the figure is not emitted.