Speculative Prose Pipeline

Prose through the engine

Speculative design. Open, not settled — thinking and proposals, not shipped behavior. See Roadmap for what's in flight.

Status: speculative. This proposal takes over BL-8 and implements step 4 of Boxes ("Prose and Markdown").

What's wrong today

Record prose (summaries, details, captions, oracle notes, and now index.md bodies) is raw text. prose-math.ts recognises only $…$, which it hands to <notatio-out>, and MathLive's static renderer typesets that. As a result:

• ${}_pF_q(a;b;z)$ isn't math: ${ is refused so that Manipulate's ${name} placeholders survive, and every $ after it pairs with the wrong partner.
• Some spans render empty.
• Backticks, emphasis and links show as literal text.
• Example cells show StandardForm (HypergeometricPFQ([1,1],[2],0.5)), not the notation.
• MathLive's static markup isn't selectable the way text is.

The model

A piece of prose is markdown, and it reads into a box tree (Boxes): prose boxes (Cell, TextData, StyleBox, ButtonBox) with formula boxes inside them. It renders from that tree. There are two kinds of hole.

Default form is StandardForm — DisplayForm where the result is already boxes rather than a formula. TraditionalForm appears only where math notation is explicitly asked for; it isn't the prose default, since most prose isn't pure math and StandardForm is the form that stays useful for strings, lists, associations and other non-math values too.

• $…$ / $$…$$ is TeX, held. The island is FormBox(tex, "TeXForm"): TeXForm under a hold, never parsed, never evaluated, so $2+2$ stays 2 + 2 and costs nothing but KaTeX. It is not TraditionalForm: what our notation would print for the same expression can differ, and when to show which is its own question. A menu on the rendered formula may offer the opt-in treatments (parse it, evaluate it, show it as TraditionalForm or Epsil).
• ${…} is a hole, as in Observable: the reader scans for them, in running text and inside TeX alike, without a parser. ${k} is a TemplateSlot, anything else a TemplateExpression holding its Epsil; a hole in TeX leaves the formula a RowBox of TeX fragments around it. The environment fills holes (a value, a knob, a readout); a page with none renders them as written. ${} is TeX's empty group, never a hole.
• <Head …> is an expression, evaluated. A tag reads through the vdom reader (Vdom: children are the arguments, as one Epsil list; attributes are named arguments or options) into MathJSON. The page's engine evaluates it, and ToBoxes of the result, in StandardForm, decides what it becomes:
  • formula boxes typeset as StandardForm, in KaTeX, inline in the run — conventional math notation, but not TraditionalForm's textbook conventions unless asked for (a TraditionalForm(...) wrapper);
  • other boxes (Manipulate, DynamicModule, a table, a plot, anything whose boxes are layout or a component) are DisplayForm: inlined as themselves, and each renderer draws them (DOM on a page, text in a pipe);
  • non-math values (a string, a list, an association, …) still typeset through StandardForm — it reads fine for these, which is why it's the default rather than TraditionalForm;
  • a free variable with a type or range (<Symbol name="k" type="integer" range="1..9" />) becomes a tangle control, as in prose-mode Manipulate (prose.ts), and the readouts that use it are DynamicBoxes.

One function, two times

renderProse(markdown, engine, env) returns prose boxes, with realisers for each environment (Rendering Environments):

• build (SSR): boxes → HTML. KaTeX's renderToString for formulas, plain HTML for the prose, and a placeholder the client hydrates for anything live. Selectable text, no MathLive.
• runtime (live pages, notebooks, the worksheet): the same function in the browser. A live hole re-renders from its DynamicBox.
• text (outline, slug, local search, <title>, copy, no-JS readers): boxes → Unicode text (2⁵³, ₂F₁(a,b;c;z)), from whichever form was rendered (a $…$ island's own TeX, read cheaply into Unicode), with InputForm where no glyph exists. This settles BL-8's heading case.

Markdown is also a target: toMarkdown(boxes) writes CommonMark with $…$ islands, for GitHub and for docs as data.

Example cells

An example's Out is its value, so it goes the same way as a tag: StandardForm in KaTeX when its boxes are formula boxes, DisplayForm otherwise — TraditionalForm only where the example is explicitly about notation. The In cell shows the example's expression held, in StandardForm, with the menu still switching forms (Epsil, FullForm, TraditionalForm, …). MathLive loads only when a cell is edited.

Answers to BL-8's questions

1. Delimiter. $…$ stays LaTeX (held TeX). ${} (an empty group) is accepted, since placeholders are never empty. Expressions are <Head> tags, not a new delimiter.
2. Inline renderer. KaTeX for formulas; DOM for boxes.
3. Copy. text/plain gets InputForm (Epsil); text/html gets the rendered HTML with the LaTeX in a KaTeX annotation.
4. Scope. The mechanism first on reference pages (records' prose and example cells), then guides and components, then the Unicode-superscript sweep.

Order

1. The prose reader and renderer in @enumeratio/boxes: the TextCell/TextData/ButtonBox/FormBox/TemplateSlot heads, the markdown subset reader (one pass, holes included), toMarkdown, toText (Unicode) and toAscii (AsciiMath's spellings), toHtml with the TeX typesetter passed in (KaTeX, from the web layer). $…$ held as TeX; ${} fixed.
2. Reference pages render records' prose through it, at build, and example cells as StandardForm in KaTeX.
3. <Head> tags in prose: vdom reader → evaluate → ToBoxes, math or boxes.
4. Controls: free typed variables as tangle knobs, DynamicBox readouts; the prose-mode Manipulate moves onto this.
5. Guides and component pages; the BL-8 sweep.

Open

• The Cell name collision. notatio's Cell (a notebook cell) versus Wolfram's prose Cell. The proposal is to name the prose block TextCell and keep Cell.
• Where the KaTeX realiser lives. @enumeratio/boxes has no runtime dependencies. The proposal puts toKatex in components (or frontend) and keeps boxes → LaTeX in boxes.
• TraditionalForm coverage. Asking for TraditionalForm falls back to StandardForm where makeBoxes has no traditional notation for a head yet. The notation-as-data step (boxes.md §2) closes those gaps.
• When to show which form. A $…$ island is TeXForm as written; a <Head> tag is StandardForm unless wrapped. Where TraditionalForm and StandardForm differ, and which a reader wants where, still needs working through.