Skip to content

v0.9.0

Choose a tag to compare

@moeakwak moeakwak released this 04 Oct 09:19
· 45 commits to main since this release
0fb2764

This release is a round of fixes that make TeXForm safer to run on real-world input. Parsing no longer panics on any source text, reports each problem with its specific diagnostic kind at its actual source position, and reads several plain TeX constructs as TeX does instead of silently misparsing them. Input that TeXForm cannot read correctly is now rejected with a diagnostic rather than turned into a wrong tree. The Corpus profile no longer drops modifiers whose visible effect depends on the math style, so its output keeps the rendering of its input. To parse the plain TeX macros \root…\of and \buildrel…\over correctly, the release adds the u{\name} form to the ArgSpec syntax; its new argument-form variant is the only breaking change, for Rust code that matches argument forms exhaustively.

Added

  • Breaking: The ArgSpec syntax gains the until form u{\name}. It describes plain TeX delimited parameters such as \def\root#1\of#2, whose argument runs up to a terminating control word instead of being one token or one brace group, which no existing form could express. The argument collects tokens up to the first terminator outside brace groups and strips one brace layer when it is exactly one group; empty content is valid, and a missing terminator is a parse error. The terminator must be a single control word such as \of or \over, and ArgSpec validation, including for custom entries, rejects characters, control symbols, and multi-token terminators. ArgumentKind, ArgKindRef, and ArgSpecFormInfo gain an Until { close } variant, and bindings, columnar export, and CLI output name the form "until"; exhaustive matches over argument forms must handle it.
  • The knowledge base covers LaTeX text boxes and related commands, so input such as \text{\mbox{$x$}}, \raisebox{1ex}{$x$}, or \left\llbracket x\right\rrbracket parses. \raisebox and \scalebox are recognized with their optional arguments; \mbox, \fbox, \hbox, \makebox, and \parbox work in text mode as well as math mode; \label and the math environment are allowed inside text containers; \ensuremath reads its argument as math in both modes; nested \text is allowed in text mode; \emph works in math mode; and \llbracket and \rrbracket are delimiters. These records describe parsing, not renderer support, and dimension arguments still require literal lengths.

Changed

  • The ArgSpec of the command root changes from m to u{\of} m. With m, the parser read only one token or group as the degree and left \of and the radicand as sibling nodes, so unbraced degrees such as \root n_i\of{x} were misparsed and \root\of{x} was rejected. A parsed root now has a degree slot and a radicand slot, and a \root without \of is a parse error. Normalized output is unchanged for previously accepted input.
  • The ArgSpec of the command buildrel changes from m to u{\over} m. With m, \buildrel took only the first token of the annotation, and the parser used a source-text lookbehind to split the rest with an infix \over, so a\over\buildrel b\over c was rejected as an ambiguous infix and a braced \over inside the annotation was mistaken for the separator. A parsed buildrel now has an annotation slot that runs to the first \over outside braces and an operator slot, and a \buildrel without \over is a parse error. Scripts after the operator attach to the whole relation. The Faithful rule base/buildrel-expand is replaced by the Authoring rule base/buildrel-to-stackrel, which rewrites \buildrel A\over B to \stackrel{A}{B}; rule selections that name the old key should use the new one. Faithful, Corpus, and Equiv output is unchanged for previously accepted input, and Authoring now succeeds instead of failing its rewrite contract.
  • Corpus and lower profiles keep \dfrac, \tfrac, \limits, and \nolimits. Their visible effect depends on the math style, which a single formula does not know, so dropping them could change the fraction size or move the scripts. Equiv still rewrites them.

Fixed

  • Source text containing a character the lexer cannot tokenize, such as an ASCII control character other than tab, newline, carriage return, or form feed, returns an invalid-character diagnostic instead of panicking. In Python the panic surfaced as PanicException, which escaped ordinary except Exception handlers; callers now receive a regular ParseError.
  • Diagnostics name the specific problem. Internal parser branch failures such as not a command no longer replace the real diagnostic, an environment left open at end of input reports unclosed-environment with \end{name} as the expected token, and an environment used in a disallowed mode reports environment-mode-error in both strict and lenient parsing. ParseDiagnosticKind in Python and TypeScript includes the new kinds.
  • Diagnostics point at the actual source position. With reject_unknown, \frac{\bad}{x} reported the unknown command at \frac, and a+\bad at a; spans inside argument content were also shifted after a comment or a run of spaces. Errors now keep the failing token's span, and diagnostics and node spans inside arguments map token by token back to the source.
  • Plain TeX alignment macros (\matrix, \pmatrix, \cases, \eqalign, \eqalignno) split rows at \\ and \newline as well as \cr, and drop a final row terminator that has no * or spacing. \eqalignno no longer fails unless every row ends in &(…): it becomes align*, and only rows with a numbering cell get a \tag (or \tag* when the cell is not wrapped in parentheses).
  • A script at the start of an alignment row attaches to an empty base in that row instead of to the preceding row separator: \begin{matrix}a\\_i b\end{matrix} no longer serializes as \\ _{i} b.
  • In array, darray, eqnarray, and eqnarray*, \\ [-5pt] reads the bracket as row spacing, as LaTeX does. amsmath environments still read it as row content, so a row can start with an interval such as [a,b].
  • An unbraced \left…\right or $…$ argument keeps its delimiters: \frac\left(x\right)2 parses like \frac{\left(x\right)}{2} instead of \frac{x}{2}, and \text$x$ like \text{$x$}.
  • A comment between a control word and following letters inside an argument no longer merges them: \sqrt[\le% comment followed by q]{x} on the next line was read as \sqrt[\leq]{x}.
  • Custom signatures with delimiters such as d** or r~~ match *, $, &, #, ^, _, and ~ in the input instead of always treating the argument as absent. ArgSpec validation rejects delimiters that can never match: %, ', ’, and ASCII control characters.
  • \hspace*{…} parses in math and text modes.
  • A text environment whose body ends with a group, inline math, or whitespace before \end no longer reports a spurious diagnostic.