Repository navigation
Speculative AsciiMath Boxes
Speculative. A proposal, not adopted.
AsciiMath is read as TeX and then as an expression, while MathML is read as boxes and then as an expression. The Ideas entry says the TeX leg "loses what TeX cannot say" and proposes a reader onto boxes. This page first checks that premise against the tree, then designs the reader, its symbol table, its writer counterpart and its tests.
"Measured" means read from the tree at origin/main (a file and line, or a count of what I read); where I read the pinned asciimath2tex 1.5.0 package's dist/asciimath2tex.modern.js (minified, so no line numbers) I name the method. "From memory, not checked" marks statements about the AsciiMath language or about compute-engine's LaTeX parser that no file in the repo backs. "Assumed" is reasoning I did not run. Nothing here was built or run: the tree has no node_modules, so every claim about what an expression reads as through compute-engine's parser is assumed unless a committed golden pins it.
Option B, narrowed: write parseAsciiMath in @notatio/boxes beside parseMathML, driven by one symbol table that the existing toAscii writer also takes its spellings from, and route fromAsciiMath through makeExpression with the same TeX fallback MathML already has. Keep asciimath2tex as a dev dependency of formats only until a differential test agrees, then drop it.
The premise holds in shape and is weaker in size. AsciiMath does reach an expression only through TeX, and MathML does take the boxes route. But the committed round-trip golden loses one of 32 corpus rows through the TeX leg (a binomial), so today's measured loss is small. The case for the reader is maintainability and a shared table: the writer's spellings are a hard-coded list the reader cannot see, and a scan shows the writer emits glyph spellings the current reader's table lacks. It is not a correctness emergency. If the owner wants only correctness, option A (document the losses) costs nothing and is enough.
First slice: the constructs in the existing 32-row golden, in packages/boxes, tested by that golden's expressions plus a new box-tree golden (see "Plan for a builder").
AsciiMath out (boxes to text).
-
toAscii(box)linearises boxes with the ASCII alphabet (packages/boxes/src/render/text.ts:217-218);toAsciiMath(json)istoAscii(makeBoxes(json))(packages/formats/src/asciimath.ts:15). - The glyph-to-spelling list is one hard-coded record,
ASCII, inpackages/boxes/src/render/linear.ts:8-119, withspellandjoinSpelledusing it (linear.ts:127-143). - The format is registered with
encode: toAsciiMathanddecode: fromAsciiMath, extension.am(packages/formats/src/formats.ts:103-111).
AsciiMath in (text to expression).
-
fromAsciiMathisce.parse(asciiMathToTeX(text)).json(packages/formats/src/asciimath.ts:27-31).asciiMathToTeXis theasciimath2texlibrary'sparse(asciimath.ts:22-27), declared atpackages/formats/package.json:71andpnpm-workspace.yaml:14. compute-engine's LaTeX parser does the rest. No reader onto boxes exists, not even in part: the onlyparseXxxthat returns boxes isparseMathML(packages/boxes/src/render/mathml.ts:413) and the markdown reader. - The header of
asciimath.ts:1-6already names this idea. - Callers: the CLI's
:asciimathpragma and-i asciimath(packages/cli/src/engine.ts:391), the format registry, and the formats test (packages/formats/tests/asciimath.test.ts). I found no input field inpackages/componentsorpackages/frontendthat reads AsciiMath; their hits are output only (packages/components/src/notatio-out.ts:745-766,packages/frontend/src/kernel-host.ts:89). The terminal environment'sform: "AsciiMathForm"(packages/frontend/src/environment.ts:84) is an output setting.
MathML in, the template.
-
fromMathMLisboxesToExpression(parseMathML(xml), ce)(packages/formats/src/mathml.ts:31-32).boxesToExpressiontriesmakeExpressionwith the registered notation, and only onMakeExpressionErrorwrites the boxes as LaTeX and lets compute-engine read that (mathml.ts:17-27). So the Ideas entry's "straight to an expression" is accurate, with a TeX fallback for declined boxes. - The wiki's Syntax-and-Formats still says MathML reaches an expression "through boxes ... and compute-engine's LaTeX parser, until MakeExpression reads boxes directly" (lines 26-30); that is out of date against
mathml.ts.web/reference/formats/asciimath.mdsays AsciiMath is "output, not input", which is true of compute-engine and false of notatio's registry. -
parseMathMLmaps elements to boxes one for one:mi/mn/moto a trimmed string token with no classification,mrowtoRowBox,msup/msub/msubsupto the script boxes,mover/munder/munderoverto the over/under boxes,mfractoFractionBox(line thickness 0 givesFractionLine: false),msqrttoSqrtBox,mroottoRadicalBox,mtabletoGridBox,mtexttoTextBox, unknown elements to their content (mathml.ts:348-411). The box constructors arepackages/boxes/src/box.ts:234-263. - Malformed MathML throws
MathMLSyntaxError(mathml.ts:238); the XML is the only thing it rejects.
makeExpression (packages/boxes/src/read.ts).
- It is a pure function of the box tree: structure by box kind, a row's tokens by operator precedence, an
InterpretationBoxas the expression it carries, and a throw ofMakeExpressionErrorfor what it cannot read, never a guess (read.ts:1-11,1028-1042).ErrorBoxandFormBox(..., "TeXForm")are declined (read.test.ts:200-212,read.ts:810-811). - Juxtaposition: a row with no operator between factors is a product (
startsFactor,read.ts:490-499). A name followed by(is a call only when it is a known function name, matches/^[fgh]$/, is in theheadsoption, or is in the registered notation (read.ts:98,543-551); any other name followed by a parenthesis is a product (assumed fromcallable, not run). A known function name takes a bare argument (sin x,read.ts:530-534);sin^2 xisPower(Sin x, 2)(power,read.ts:820-834). - Fences:
|x|isAbs(a likelier head is chosen overNorm,read.ts:8-9);⌊ ⌋isFloor(read.ts:740);[/(over aGridBoxisMatrix(read.ts:686-689,1006-1011);{over a grid ofvalue, if conditionrows isWhich, where the condition cell starts withTextBox("if ")or isTextBox("otherwise")(read.ts:690-693,1013-1026;makeBoxeswrites exactly that atmake.ts:543-547); aFractionLine: falsefraction in a fence is a binomial (read.ts:694-697). - Big operators:
UnderoverscriptBox("∑" | "∏", under, over)andUnderscriptBox("lim", under)read asSum/Product/Limit;SubsuperscriptBox("∫", lo, hi)asIntegrate(read.ts:783-806). A reader that emits stacked limits forsumandprodand sub/superscript forintmatches whatmakeBoxeswrites (make.ts:343-344). - The MathML tests already pin juxtaposed tokens:
2x+y,sin x,f(x,y),|x|+1,∫x²dx, chained relations asAnd(packages/boxes/tests/read.test.ts:76-103). An AsciiMath reader emitting the same token rows inherits these.
What the round trip loses today (measured). packages/formats/tests/asciimath.test.ts prints each of 32 corpus expressions (packages/formats/tests/corpus.ts) as AsciiMath, reads it back, and records only rows that differ. One row differs: \binom{5}{3} prints (5; 3) and reads back as ["Tuple", 5, 3] (asciimath.golden.json). The matrix row ((1, 2), (3, 4)), sum_(n = 1)^10 n^2, int_0^1 x^2 dx, lim_(x -> 0) (sin(x))/x, root(3)(x) and log_2(8) all read back equal. So the corpus is too easy to show what the TeX leg loses; it shows only that the common constructs survive.
The TeX leg's shape (measured by reading asciimath2tex 1.5.0).
- It never throws: the class has an
errormethod that nothing calls (zero call sites), and an unmatched character falls througharbitrary_constantinto the output unchanged. - Letters are single-character names;
sin,cos,log,ln,exp,gcd,detand a few more, and the single lettersfandg, are "function" unary symbols (his not, unlikeread.ts:98).abs,norm,floor,ceilbecome\left | … \right |,\left \| … \right \|,\left \lfloor … \right \rfloor,\left \lceil … \right \rceil. - A bracket pair is stripped to a bare group as a fraction or script operand:
(x+1)/(y-2)becomes\frac{x + 1}{y - 2};x_(i+1)^(2n)becomesx_{i + 1}^{2 n}(itsunbracket, applied to(,[,{pairs only). - A bracketed list of two or more rows or columns of the same width is a matrix,
\left [ \begin{matrix} … \end{matrix} \right ];{ (…), (…) :}is the same with\left \lbraceand\right .;ifis a symbol,otherwiseis not (it reads as letters). -
"…"becomes\text{…};sum,prod,lim,intbecome their TeX commands with_{}/^{}.
The "TeX route" column is what asciimath2tex emits (measured, above) followed by an assumed compute-engine reading, except where a golden row pins it. The "boxes reader" column is the box tree this design emits and what makeExpression does with it (measured for the reading, designed for the tree).
| Input | TeX route | Boxes reader emits, and makeExpression reads |
|---|---|---|
f(x), sin x, sin(x)^2
|
f{\left ( x \right )}, \sin{x}, \sin{\left ( x \right )}^{2}; braced arguments are an assumed risk in the LaTeX parser |
row f ( x ) is f(x) (read.test.ts:85); sin x is Sin(x); sin(x)^2 depends on a precedence question (open question 4) |
abc, xy, speed
|
letters apart, a b c; same as AsciiMath's own reading (from memory, not checked) |
one token per letter, a product; a multi-letter symbol has no spelling (open question 7) |
abs(x), ` |
x | ` |
floor(x), ceil(x)
|
\left \lfloor x \right \rfloor (assumed readable) |
row ⌊ x ⌋, Floor (read.ts:740) |
[[a,b],[c,d]] |
\left [ \begin{matrix} a & b \\ c & d \end{matrix} \right ]; the parser's reading of a bracketed matrix is assumed; a paren matrix round trips (golden) |
[, GridBox, ], read as Matrix with the bracket kept (read.ts:1006-1011) |
{(x, if x>0),(0, "otherwise"):} |
a matrix with \lbrace; Which is not recoverable from it (assumed) |
{ over GridBox with TextBox("if ") and TextBox("otherwise") cells: Which (read.ts:1013-1026) |
x_(i+1)^(2n) |
x_{i + 1}^{2 n}; no loss expected |
SubsuperscriptBox(x, row(i + 1), row(2 n)) with the parentheses stripped |
sum_(i=1)^n i^3 |
\sum_{i = 1}^{n} i ^{3} and the golden shows this family reads (asciimath.test.ts:37-43) |
UnderoverscriptBox("∑", row(i = 1), n) then i^3; Sum (read.ts:793) |
int_a^b f(x) dx |
dx is a constant symbol; golden row int_0^1 x^2 dx round trips |
SubsuperscriptBox("∫", a, b), then f ( x ) d x with d and x separate, as in read.test.ts:88-91; Integrate
|
root(3)(x) |
\sqrt[3]{x}; round trips (golden) |
RadicalBox(x, 3), Root (read.ts:775) |
| a binomial | AsciiMath has none; the writer prints (5; 3) and the TeX route reads a Tuple (golden) |
not readable from AsciiMath unless we extend it (open question 5) |
"text" |
\text{text} |
TextBox (a TextBox reads as a string, read.ts:764) |
->, <=, a<b<=c
|
\to, \le; chains read as And through the engine (assumed) |
tokens → ≤; chained relations are And (read.test.ts:81-85) |
Unicode input α ≤ π
|
passes through arbitrary_constant; reading depends on the parser's Unicode support (unicode-tex.ts:1-5 lists what it reads) |
leaf tokens as typed; no dependence on TeX |
| a symbol with no TeX command | falls to a literal character | a token; makeExpression reads a name it knows or leaves a symbol |
an unfinished (x +
|
emits \left( x + \right. and the engine's reading is assumed |
row(["(", "x", "+"]); shown, and makeExpression declines it (read.test.ts:204-206) |
What the table says: the cases that differ in meaning are the ones where the engine's LaTeX parser must guess from layout that a box pair means Abs, Floor, Matrix or Which, and where TeX has no word for the meaning (Which, a binomial). Everything else reads the same on both routes. The strongest concrete difference is the piecewise case, and it is assumed, not measured, because I could not run the parser.
What the writer emits that the current reader cannot read (measured by a scan of the table in the installed package against linear.ts:8-119). Twelve spellings are not in the reader's tables: -+ (∓), ni (∋), <- (←), <-> (↔), || (‖), PP (ℙ), hbar (ℏ), deg (°), varpi (ϖ), varrho (ϱ), varsigma (ς), Upsilon (Υ). They become separate characters or letters on the TeX route. Three pairs of glyphs share one spelling, so reading cannot recover which was written: ≤ and ⇐ as <=, ≃ and ≅ as ~=, ⋅ and · as * (linear.ts:12-13, 19, 25-26, 47). The scan was a regular expression over minified code; the builder should recheck it by running the table.
Where. packages/boxes/src/render/, as asciimath.ts, exporting parseAsciiMath(source): Box next to parseMathML and re-exported from render/index.ts (render/index.ts:17). boxes has no engine and no DOM (wiki Boxes, line 179); a reader needs neither. formats/asciimath.ts then becomes boxesToExpression(parseAsciiMath(text), ce), which means boxesToExpression moves out of formats/mathml.ts into a shared module (it is the only thing both need).
One table. Replace the ASCII record in linear.ts with a list in the style of UNICODE_TEX (unicode-tex.ts:30-47): one readonly list of { char, ascii: readonly string[] }, where the first spelling is what the writer prints and every spelling is read. UNICODE_TEX is the pattern the repo already uses to keep a writer and its readers in step, and it carries a note per entry on why a symbol is left out. The shared list makes the three collisions above explicit: a reader reads <= as ≤, and the writer spells ⇐ as lArr (that spelling is in asciimath2tex's table; the choice for the AsciiMath language is from memory, not checked).
Tokenizer. Longest match from the table at each position, then these classes (all but the first are from memory of the published grammar, not checked): a number \d+(\.\d+)?; a quoted string "…" as TextBox; a single letter as a one-character identifier; a symbol word (sum, alpha, oo, RR, sin) from the table; a punctuation run from the table (<=, ->, |->, **, (:, {:); any other character as itself, so Unicode passes through. Spaces separate tokens and are dropped.
Parser. Recursive descent, emitting boxes directly; no AST in between. The grammar is small (from memory, not checked): a simple expression is a symbol, a bracketed group, a unary construct applied to one simple expression (sqrt, abs, floor, ceil, norm, hat, bar, vec, text, font switches), or a binary construct applied to two (frac, root, stackrel, overset, underset); an intermediate expression is a simple expression with _ and/or ^; and / joins two intermediates into a fraction. Emission:
- Matched brackets become a nested
RowBoxwhose first and last items are the bracket tokens, asmakeBoxeswrites a fence (make.ts:62-64,305), so a script after a group attaches to the whole group;makeExpressionreads both nested and flat fences (read.ts:521-546). -
/and the scripts strip one outer(…),[…]or{…}pair from an operand that is exactly that pair:(x+1)/(y-2)isFractionBox(row(x + 1), row(y - 2)).asciimath2texstrips only those three pairs, and not|…|or⌊…⌋(measured).{:and:}are invisible fences, not tokens. -
sqrt xisSqrtBox,root(n)(x)isRadicalBox(x, n)(box.ts:244-245).abs,floor,ceilandnormemit the fence tokens|,⌊ ⌋,⌈ ⌉and‖, whichmakeExpressionreads (read.ts:736-743). -
sum,prodandlimwith_and^emitUnderoverscriptBoxandUnderscriptBox;intandointwith scripts emitSubsuperscriptBox/SubscriptBox; scripts on anything else emitSuperscriptBox,SubscriptBox,SubsuperscriptBox. These follow whatread.ts:777-806reads. - A bracketed list whose elements are all same-width bracketed lists, with more than one row or column, becomes a
GridBoxinside its fence, usingasciimath2tex's rule (measured: matrix when columns > 1 or rows > 1 and every row has the same width). A{fence over rows of two cells, with a second cell led byifor"otherwise", emits theTextBoxshapespiecewisewants (read.ts:1013-1026). A bare(a, b)stays a row. - A symbol word that is a known function stays a plain token (
sin);makeExpressiondecides call versus product. The reader emits noApplyFunction(U+2061) and noInvisibleTimes(U+2062): AsciiMath has no such characters andmakeExpressiondoes not need them (read.test.ts:78-86reads MathML without them, apart from the one case that carries U+2061).
Errors. The parser is total, like asciimath2tex (measured: it does not throw), but marks what it could not use instead of passing it through: an unmatched closer, a unary construct with no operand, or a tail that is not a valid expression becomes ErrorBox(token) in the row. An unclosed opener stays a bare ( token with no closer. This keeps a live field showing partial input, and it keeps showing separate from meaning: makeExpression already declines row(["(", "x"]), row(["x", "+"]) and ErrorBox (read.test.ts:200-206). The formats layer should surface that decline for a parse error rather than falling back to TeX (open question 2).
It exists: toAscii (text.ts:217-218). What it does and does not do, measured from text.ts and linear.ts:
- Invisible times writes as a space and
ApplyFunctionas nothing (text.ts:47-48), so boxes to AsciiMath to boxes cannot be an identity. The property to test is equality aftermakeExpression, which is how the existing golden reads it (asciimath.test.ts:22-27). - Spelled correctly as AsciiMath: scripts with
^( )and_( ), fractionsa/bwith parentheses where needed,sqrt(x),root(n)(x),sum_(…)^…, a matrix in a bracket fence as[[1, 2], [3, 4]], a closed-interval call, a repeating decimal'sbar(…)(text.ts:75-134). - Written in notatio's own spelling, which is not AsciiMath: a piecewise function as
{value if cond; value otherwise}(text.ts:85-87, no quotes unlessShowStringCharacters), a plainGridBoxas{{…}, {…}}(text.ts:134-138), a fraction with no line asa; b(text.ts:127-128), an integer range as aRange(…)call (text.ts:98). The reader above cannot read these back. - Written as a placeholder, silently: graphics
-Graphics-and-Graphics3D-, controls-Slider-and the like,-TableView-,-Dynamic-; a TeXFormBoxgoes throughtexToAlphabet(text.ts:190-215). The writer never declines. - Multi-letter names are spaced from a neighbouring letter (
joinSpelled,linear.ts:134-143), butspeedalone reads back as five letters.
Proposal for the writer: keep the placeholders and say so in the doc comment; in slice 2, change the piecewise spelling to {(value, if cond), (value, "otherwise") :} and the plain grid to a bracketed matrix so the reader reads them; keep every other spelling. A box with no AsciiMath spelling stays a placeholder (the round-trip test lists which heads, so a new head cannot pass unnoticed).
Who types AsciiMath: I found only the CLI pragma and the registry's .am decode as callers (see What exists). Whether anyone uses them is unmeasured. The project's own text syntax is Epsil, with $…$ LaTeX islands (AGENTS.md "Names"; Syntax-and-Formats lines 15-33); AsciiMath is a format whose output half is the one in use (it is the text/plain fallback, packages/frontend/src/ipynb.ts:31).
| Option | Cost | What it breaks or leaves |
|---|---|---|
| A. Keep the TeX leg, document the losses | One doc page; no code | The writer/reader table drift stays (12 unreadable spellings, 3 collisions); Which and similar remain unreachable |
| B. Boxes reader (this page) | A reader of a few hundred lines (estimate, assumed), one shared table, a test corpus; drop a dependency | Reads fewer unusual inputs than a mature library until the table is complete; precedence questions (open question 4) must be settled by a person; a regression is possible on inputs the TeX leg reads now, which the TeX fallback covers when makeExpression declines |
| C. Drop AsciiMath input, keep output | Delete fromAsciiMath, the dependency, the CLI case, the registry decode
|
Breaks :asciimath and -i asciimath and the formats test's read half; removes the format's retypeable use, and Syntax-and-Formats' statement that it reads and writes |
| D. Read AsciiMath by translating to Epsil text | A translator that needs the same heuristics as makeExpression (what ` |
x |
Recommend B. The writer and its table are already in boxes, makeExpression already carries the semantics a reader needs, and the reader is then a thin parse onto existing heads. D is dominated by B; C is the choice only if the owner knows nobody reads AsciiMath.
First slice, in packages/boxes only, no formats change:
- Move the
ASCIIrecord inlinear.tsinto a listASCIIMATH_SYMBOLSof{ char, ascii[] }in its own file, first spelling written, all read. Leave the writer's output unchanged (the existingboxes.golden.jsonasciicolumn pins this:boxes.test.ts:50-66). Fix the three collisions by choosing the first spelling per glyph and noting it. - Add
packages/boxes/src/render/asciimath.tswithparseAsciiMath, supporting exactly: numbers, letters, symbol words and punctuation from the table, quoted text, bracket groups,_and^with bracket stripping,/,sqrt,root,frac,abs,floor,ceil,sum,prod,lim,int, matrices, piecewise, andErrorBoxfor the unusable. Export it fromrender/index.ts. - Test (new file
packages/boxes/tests/asciimath-read.test.ts):- a golden of about 40 AsciiMath strings to
{ boxes, expression }, each string chosen from the table in section 1, behindUPDATE_ASCIIMATH_BOXES=1, compared withtoEqual, notoMatchSnapshot; - the boxes
CORPUS(packages/boxes/tests/corpus.ts, about 180 expressions by my count of its entries) throughmakeBoxes,toAscii,parseAsciiMath,makeExpression, classified asexact,equivalent,declined,mismatchedwith explicit expected lists, the shape ofread.test.ts:56-66; expected extras includespeed, the piecewise spelling and the plain grid until slice 2; - a check that every glyph in
ASCIIMATH_SYMBOLStokenizes back to itself.
- a golden of about 40 AsciiMath strings to
- Slice 2, in
formats: pointfromAsciiMathatboxesToExpression(parseAsciiMath(text), ce); keepasciimath2texas a dev dependency and add a differential test overpackages/formats/tests/corpus.tsand the new strings that asserts agreement at expression level where the TeX route is right (thebackrows in the golden record where it is not); once it agrees, drop the dependency,asciiMathToTeXand its export (formats/src/index.ts:8). Then fix the writer's piecewise and grid spellings. - Update the stale lines in Syntax-and-Formats (MathML path) and
web/reference/formats/asciimath.mdin the same change.
-
Where does the reader live:
boxesorformats? Recommendboxes/render/, besideparseMathML, because it is pure, needs no engine, and shares a table withlinear.ts. -
Does a parse error fall back to TeX like a declined box? Recommend: a decline (
MakeExpressionError) falls back, as MathML does (mathml.ts:24-27), so nothing that reads today stops reading; a box tree containingErrorBoxsurfaces the error and does not fall back, because the TeX leg would hide it. -
Which grammar is the reference? Recommend asciimath.org's published grammar and symbol table. I did not fetch it; the builder should check the symbol list against it, since
asciimath2tex's table is one library's reading of it. -
Does
sin(x)^2mean(sin x)^2orsinof(x)^2?asciimath2texputs the script on the wholesin(x)(measured,intermediateoverunary). The language's own grammar may say otherwise (from memory, not checked). Recommend(sin x)^2, confirmed against the reference implementation, with a golden row. -
Do we extend AsciiMath with a binomial? Recommend no in slice 1; the binomial stays the golden's one known gap. If wanted,
binom(n)(k)as a binary construct emitting aFractionLine: falsefraction in a fence, marked as ours. - Do we change the writer's piecewise and grid spellings? Recommend yes, in slice 2, so output reads back; they are not valid AsciiMath today.
-
Multi-letter names. AsciiMath has one letter per symbol, so
speedreads as a product. Recommend leaving it: the writer'sspeedthen lands in the expected-mismatch list, and a symbol that needs a long name is written in quotes only if the owner wants quotes to mean a symbol (they mean a string today). -
Chained
/(a/b/c):asciimath2texleaves the second/as a bare character (measured, by readingintermediate_or_fraction). Recommend anErrorBoxon the second slash until the grammar is checked. -
Dropping
asciimath2tex. Recommend dev-only until the differential test agrees, then remove.
Measured, from the tree: the paths, callers and golden above; makeExpression's contract, juxtaposition and fence handling, and its declines; parseMathML's element map; toAscii's spellings, collisions and placeholders; the asciimath2tex structure and symbol tables (from the installed 1.5.0 distribution, by reading the minified source and a regular-expression comparison with linear.ts); the wiki and web pages' stale statements; the corpus sizes (32 rows in the formats corpus by its file; the boxes corpus about 180 by a line-pattern count).
From memory, not checked: AsciiMath's grammar as the bullets in section 2 give it, the reference behaviour of sin(x)^2 and of chained /, the spelling lArr, and that compute-engine's LaTeX parser has no braced-argument or \lfloor problem.
Assumed: every claim in the "TeX route" column that no golden row pins, the size of the reader, who uses AsciiMath input, and that the 12 unreadable spellings behave as separate characters (the code path says so; I did not run it).