Skip to content

engine: the cell scanner counts braces the way align_state does - #154

Merged
tannevaled merged 1 commit into
mainfrom
align-state-braces
Aug 31, 2026
Merged

engine: the cell scanner counts braces the way align_state does#154
tannevaled merged 1 commit into
mainfrom
align-state-braces

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

#153 said "a matrix is not braced". That is true of this engine and false of TeX, and the difference is worth stating where the code is read.

An alignment entry ends when a tab or \cr occurs, provided that the current level of braces is the same as the level that was present at the beginning of that alignment entry; i.e., provided that align_state has returned to the value it had after the ⟨u_j⟩ template.
— tex.web §7259-7264, with align_state ±1 per brace at §6738-6742

A nested matrix is protected by a brace in TeX — an implicit one:

\newenvironment{bmatrix}{\left[\env@matrix}{\endmatrix\right]}   % amsmath.sty:1082
\def\env@matrix{…\array{*\c@MaxMatrixCols c}}                    % amsmath.sty:1058
\def\@array[#1]#2{…\bgroup …}                                    % latex.ltx:12101

This engine never expands \bmatrix — the maths layer parses the source text itself — so that \bgroup never arrives and the counter cannot see it. Counting the environment is the same protection by another road, and the comment now says that instead of a false statement about TeX.

The reading also turned up something the code did not do. tex.web §7492-7493 counts implicit braces — \bgroup and \egroup — in align_state. The cell scanner looked only at category-1 and -2 characters, so a & inside \bgroup…\egroup cut a cell TeX would not have cut. implicitChar is what the box scanner already uses for \bgroup; the cell scanner now uses it too.

Measured — 200 arXiv papers, 200 beamer talks

Not one document changes. The corpus does not exercise it; this plugs a gap rather than fixing a visible defect, like #117 before it.

Test

The unit test drives the scanner directly, since & must carry category 4 for any of this to mean anything (tokenizeTeX makes it an ordinary character): with a &= \bgroup x & y\egroup a row has two cells, and on main it has three.

#153 said "a matrix is not braced". That is true of this engine and FALSE of TeX,
and the difference is worth stating where the code is read. tex.web §6738-6742:
align_state is increased by 1 for a { and decreased by 1 for a }, and §7259-7264: an
alignment entry ends at a tab or \cr only when align_state is back where it started.
A nested matrix is protected by a real brace, an implicit one:

	\newenvironment{bmatrix}{\left[\env@matrix}{\endmatrix\right]}   amsmath.sty:1082
	\def\env@matrix{…\array{*\c@MaxMatrixCols c}}                    amsmath.sty:1058
	\def\@array[#1]#2{…\bgroup …}                                    latex.ltx:12101

This engine never expands \bmatrix — the maths layer parses the source text itself —
so that \bgroup never arrives and the counter cannot see it. Counting the
environment is the same protection by another road, and the comment now says so.

The reading also turned up something the code did not do: tex.web §7492-7493 counts
IMPLICIT braces, \bgroup and \egroup, in align_state. The cell scanner looked only
at category-1 and -2 characters, so a & inside \bgroup…\egroup cut a cell that TeX
would not have cut. implicitChar is what the box scanner already uses for \bgroup;
the cell scanner now uses it too.

Measured over 200 arXiv papers and 200 beamer talks: not one document changes — the
corpus does not exercise it. The unit test does, on the scanner directly: with
\bgroup x & y\egroup a row has two cells, and had three.
@tannevaled
tannevaled merged commit 444845b into main Aug 31, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant