Speculative Native Rendering

Design: native rendering from boxes

Speculative design. Open, not settled — thinking and proposals, not shipped behavior. See Roadmap for what's in flight.

Status: research, nothing built. Idea (the maintainer, 2026-10-06): realize our standard Boxes natively, on mobile, with no lit elements. Companion to Speculative Rendering Environments (the axes) and Rendering Environments (what landed).

This is general layout, not math layout. Math is one client. Combinatorics produces shapes that look nothing like formulas (tableaux, paths, trees, diagrams, tilings); collections want tables and lists; graphics want scenes; prose wants rich, formatted, sometimes interactive text. Boxes, the generic graphics primitives and the vdom together are a general, semantic markup: notatio markup should be useful to someone (or some agent) with no interest in mathematics at all, a durable way to mark up meaning and its presentation, the thing HTML set out to be. Every choice below is judged against that, not against typesetting formulas.

The short version: boxes are the one thing every environment can draw, so make them the only contract. Each environment gets a small primitive library that draws boxes with its own dependencies; generic components lower to boxes; a registry lets a library hint per environment. The hard part is layout in general (math, tables, grids, structures, rich text, graphics), and it is one layout engine we write once, pure and measured through a callback, not a dependency per target.

1. What exists today

Boxes (packages/boxes)

• Primitives (src/box.ts, BoxNode): RowBox, TextBox, the script boxes (Super/Sub/Subsuper/Over/Under/Underover), FractionBox, SqrtBox, RadicalBox, GridBox, StyleBox, FrameBox, TagBox, InterpretationBox, ErrorBox; prose (TextCell, TextData, ButtonBox); FormBox (TeX held as written); TemplateSlot / TemplateExpression (holes the environment fills). A leaf is a token string, classed by one lexer (tokenClass). Options are a trailing object (FractionLine, Background, FontSize, FontWeight…).
• Serialisers (src/render/): toLatex, toMathML/parseMathML, toText/toAscii (one line, linear.ts), toHtml (prose only, typesetter passed in), toMarkdown/readMarkdown. All are boxes to a string; none lays anything out. There is no 2-D text serialiser yet (Boxes "Next" item 5).
• Not boxes yet (Boxes "Still to come"): GraphicsBox, DynamicBox, TemplateBox, the layout formatters Row/Column/Grid/Panel/Labeled as makeBoxes rules, AdjustmentBox/PaneBox/PanelBox, TooltipBox; a control box (ButtonBox is the only interactive box and is prose-only).
• Layering: boxes is base (no DOM, no engine at runtime); /render is presentation (packages/boxes/package.json).

Components, controls, plots

• notatio-out typesets via makeBoxes then toLatex into MathLive/KaTeX (packages/components/src/notatio-out.ts, mathlive.ts). So the web draws boxes only by way of LaTeX, not by toMathML.
• A head that draws is looked up in VISUAL_SYMBOLS and lowered to a Rendering {tag, attributes, children} (packages/frontend/src/symbols.ts, renderingOf); the tags are Lit elements (packages/components/src/notatio-*.ts). This path bypasses boxes entirely: plots, charts, the Control family, Manipulate, Dynamic, Row/Column/ Grid/Panel/Labeled, the notebook.
• Plots and glyphs are pure SVG-string functions (frontend/src/plot.ts, glyphs.ts, primitives.ts, complex-plot.ts); ComplexPlot wants WebGPU, with a CPU path (gpu-eval.ts) and @enumeratio/raster (resvg, Node) for PNG.
• Controls share one contract (frontend/src/controls.ts): name, a MathJSON binding, notatio-control-change events. The arithmetic of scrubbing and playback is DOM-free (scrub.ts, playback.ts, reactive.ts). Declarations are readable without a DOM (declarations, pin, sampleValues in reduce.ts).

Vdom and markup

structuralOf / vdomOf / toVNode (frontend/src/vdom.ts) turn MathJSON into a {tag, props, children} tree for any framework's h; @enumeratio/formats/markup reads and writes it as JSX. Framework-neutral, but its tags are notatio-* elements, so a native host would mount the same Rendering with native views instead — the vdom is the expression tree, boxes are what an element draws inside itself (Boxes "The link back to meaning").

Environments and the terminal

• Environment (frontend/src/environment.ts): engine, interaction, time, surface, form, layout, colour, theme, static policy; presets WEB, PRINT, TTY, PIPE, COMPACT. reduce(expr, env) (reduce.ts) rewrites in the expression what the record cannot honour (pin/sample controls, rasterize GPU plots, stack rows).
• The TUI is a mount over Rendering, not boxes: cli/src/drive.ts (keyed sliders strip, body pinned per keypress), present.ts, textual.ts + frontend/src/textplot.ts (braille plot). Typeset math at a terminal is toText/toAscii one-liners; images via kitty/iTerm (cli/src/node-graphics.ts).

Gaps a native renderer exposes

need	today
math layout (frac, radical, scripts, grid, limits)	boxes exist; layout only inside MathLive/KaTeX/MathML
text styling	StyleBox options, drawn by LaTeX/MathML only
layout containers	Rendering tags, not boxes
graphics	SVG strings, not boxes
controls, readouts	Rendering tags + DOM events, not boxes
prose	boxes (TextCell…), HTML/Markdown only
theming	CSS custom properties inside SVG strings

2. Native options

All claims below were checked 2026-10-06 against the cited page; "unverified" marks gaps.

NativeScript

• Model: real native views driven from JS; one JS thread, no web view. Core 9.x (9.1 post) is on V8 14.9 with Vite 8 HMR and lists Vue, Angular, React and Solid; Svelte is "future work" there. NativeScript ships Node-API engine previews with Hermes, QuickJS and JSC as options (8.9 previews); the 9.1 post names only V8 — which engine is default and supported is unverified.
• nativescript-vue: Vue 3 is GA (repo); small project (~6.5k stars, ~140 commits on main), community-run. Last-commit date and Vue 3.5/3.6 compatibility unverified. Vue is our stack's side (the site is VitePress/Vue).
• JS/TS logic: V8 means BigInt, modern JS, structuredClone, TextDecoder, workers (9.1 post). WebAssembly: not mentioned in the 9.1 post, V8 supports it, NativeScript exposing it is unverified (spike item). Compute-engine and our packages should run as plain ESM.
• Native APIs: direct access to UIKit/Android SDK from TS — the only option here where we could call CoreText/StaticLayout for measurement, or SwiftMath/AndroidMath, from JS without writing a native module.
• Size: ships a V8 snapshot; the iOS metadata alone is ~11 MB (9.1 post). Measured app size unverified.

React Native (+ Expo)

• Model: React components to native views via Fabric. 0.84 (2026-02-11) made Hermes V1 the default and added WebAssembly in Hermes (Callstack).
• Drawing: react-native-svg (15.15.x, Fabric) draws an SVG display list; @shopify/react-native-skia (2.x, in Expo) has Paragraph and glyph APIs and a GPU canvas — the better fit for plots and a math display list.
• JS/TS logic: Hermes. BigInt: a feature since v0.12 per the issue trail, JSI creation/extraction landed (#775); full conformance on V1 unverified — compute-engine's Decimal path and our native-bigint kernels must be run, not assumed. Workers: no web Worker; react-native-worklets/threads exist (unverified for our use). Intl coverage is partial (unverified current state).
• Team fit: React, which the stack uses only for the (removed) wrappers; nothing in the repo is React-native-shaped.
• Size: small bare app; Expo adds. Not measured.

Capacitor / Ionic (contrast: web view)

Capacitor 8 (8.5 on 2026-07-31, Ionic) wraps the existing web in a WKWebView / Android WebView. Zero new renderer: our lit elements, MathLive/KaTeX, WebGPU- where-available run as is; cost is feel, the web-view GPU limits, and Android WebView version drift. This is the baseline any native effort must beat, and the answer to "mobile, now". Tauri 2 mobile is the same model with a Rust shell: stable 2.11.x, mobile "newer and less mature", system webviews differ per platform (survey).

Lynx

ByteDance's framework, open-sourced 2025, v3.x, monthly releases from mid-2026 (roadmap). Web-flavoured: CSS-styled native elements, ReactLynx official, Vue Lynx community. Its own JS engine (PrimJS, QuickJS-derived — from memory, unverified) makes BigInt/WASM the biggest risk. Credible and web-shaped; young. Watch, don't spike first.

Others, briefly

Flutter/Compose Multiplatform: strong custom canvases, but Dart/Kotlin — our TS packages would run in an embedded JS engine; out. Native Swift/Kotlin hosts with a JS engine (JavaScriptCore / QuickJS) running the same boxes: the minimal-dependency end of the spectrum, most work per platform, and the honest reference implementation of the contract.

Math typesetting by option

option	how	notes
SwiftMath (iOS/macOS), AndroidMath	native views from a LaTeX string; TeX rules; bundled math fonts (SwiftMath: 12)	maintained as of 2026 per search results; LaTeX in, so we'd round-trip toLatex (loses InterpretationBox, hole structure); no hook for our hover/selection; Android port maturity unverified
MathJax/KaTeX to SVG	toLatex then SVG string, drawn by react-native-svg/Skia or NS ImageView	fidelity good; needs a DOM-less run (MathJax has mathjax-full SVG output, works in plain JS; KaTeX has no SVG output); react-native-mathjax-svg is 5 years stale
KaTeX in a web view	per formula web view	heavy, per-formula cost, no real integration
own layout over boxes	§3	no LaTeX round trip; touchable pieces

3. Layout natively (math is one case)

Boxes are already the right input: a tree with no parsing. What is missing is the function boxes to a positioned display list (glyph runs, rules, boxes, baselines), after which any surface draws it. The engine has to cover general layout, not only formulas:

• flow and rich text: paragraphs, inline runs with styles, links and inline interactive pieces, line breaking (prose boxes: TextCell, TextData, ButtonBox);
• tables and grids: GridBox with spans, alignment, rules and headers, for collections and results, not just matrices;
• combinatorial structures: tableaux, lattice paths, trees, permutation diagrams, tilings, which are grids and graphics, not formulas;
• containers: rows, columns, panels, labels, frames, panes with scrolling and priorities on narrow screens;
• graphics: the display list doubles as the graphics primitive set (GraphicsBox);
• math: the TeX rules below, as one specialization.

The math-specific choices that follow are the hardest typographic part, not the whole engine.

Option A: our layout engine over boxes. Pure TS, layout(box, style, metrics) -> DisplayList, implementing TeX's Appendix G rules (as KaTeX's buildHTML and iosMath do) for the box heads above: fraction (num/denom shift, rule thickness), radical (surd glyph size variants, vinculum), scripts (shifts from superscriptShiftUp etc.), limits, GridBox (row/column gaps), atom spacing from tokenClass (ord/op/rel classes, as the MathML operator dictionary). The constants come from an OpenType MATH table of one bundled font (Latin Modern Math or STIX Two Math), extracted at build into JSON; only metrics.measure(glyphs, font) is environment-supplied (CoreText / Paint on native, canvas on web, cell width on a TUI). Stretchy delimiters and the surd need the font's glyph variants (MATH MathVariants) — real work, and the part that most separates TeX-quality from "stacked labels". Effort: a few weeks for fractions, scripts, radicals, grids, fences; a long tail (stretchy operators, accents, \\ multi-line, line breaking) after. Fidelity: close to TeX for the covered set; identical across iOS, Android and the web fallback, which is a feature (goldens).

Option B: an existing library (SwiftMath/AndroidMath). Days to a first result, best typography for free, but per-platform native code, two separately maintained engines (one per OS), LaTeX as the interface, and our InterpretationBox, selection and hover cannot reach into it. Fine as an @enumeratio/boxes-native fallback leaf for a FormBox(tex) island.

Option C: SVG. toLatex through MathJax SVG, or our own display list serialised as SVG (the same list A produces). Cheapest and portable (react-native-svg, Skia, NS ImageView); non-selectable, no native text scaling/accessibility unless a label is added from toText.

Recommendation: A as the destination with C as its first drawer (the display list as SVG runs everywhere, including the web for a golden-tested comparison with MathML Core), B only for TeX islands we do not own. Accessibility: every drawn box also gets toText(box) as its label. The same layout with a character-cell metrics is the two-dimensional text serialiser the terminal lacks (Boxes "Next" 5), so A pays for the TUI too.

4. Architecture proposal

Layering

boxes (base)                       box tree, makeBoxes, notation, registry
  └ boxes/render (presentation)    toLatex, toMathML, toText ... (exists)
boxes-layout (base, pure)          layout(box, style, metrics) -> DisplayList; MATH constants JSON
boxes-ui (presentation, pure)      the interface boxes: layout formatters, controls, graphics, dynamics
  ├ boxes-web      (lit / DOM)     today's notatio-* elements, rebased onto boxes-ui, in time
  ├ boxes-tui      (Node, cells)   the cli drive/present/textplot code, rebased
  └ boxes-native   (NS or RN)      one library per host framework: boxes-nativescript, boxes-react-native

Names are placeholders (the rename rules in Component-Naming still apply; a library is @enumeratio/…, and a new package declares layer and extends per the hierarchy test). Constraints: the first two import no DOM and no framework; boxes-ui imports boxes and frontend's pure parts (controls.ts, reduce.ts), no environment; each environment library imports only boxes-ui, boxes-layout and its own framework. A downstream library never imports an environment library.

The boxes-ui move: components lower to boxes

Today a head lowers to a Rendering of notatio-* tags. The proposal adds a second lowering, interfaceBoxes(expr), to the interface boxes, so a head with no environment-specific form still draws:

• layout: Row, Column, Grid, Panel, Labeled as GridBox/FrameBox/PaneBox (the Boxes "Next" 3 item), so a native renderer needs no layout heads of its own;
• GraphicsBox (primitives as in primitives.ts, plots as GraphicsBox of polylines/axes): the pure SVG functions become one display-list producer, and SVG is one drawer;
• DynamicBox(expr) and a control box ControlBox(name, kind, spec, options) (kind from CONTROL_HEADS, spec the range/choices/initial); a scope owns bindings exactly as notatio-dynamic-module does now.

Not every symbol needs a native-specific component: anything unhinted lowers to boxes, and boxes always draw. A symbol that wants more (ComplexPlot on a GPU) registers a hint (below).

What a primitive library implements

interface BoxRenderer<Out> {                       // one per environment
  readonly environment: Environment;               // the existing capability record
  readonly metrics: Metrics;                       // text measure, font, em, theme resolver
  render(box: Box, ctx: RenderContext): Out;       // dispatch on box head
  readonly heads: Partial<Record<BoxHead | InterfaceHead, HeadRenderer<Out>>>;
  readonly capabilities: readonly BoxHead[];       // heads it draws itself (others get §below)
}

• Per-box-kind renderers: a head not in heads falls back, in order, to (1) a hint for it (below), (2) its reduction (a box rewrite to simpler boxes: RadicalBox to SqrtBox with a prefix, GridBox to a RowBox of rows), (3) toText(box) in a label. Boxes always draw.
• Capabilities extend, not replace, Environment: heads plus new axes the record lacks — fonts (has a MATH font), gestures (touch drag, long-press, second finger), focus (keys, none), textSelection, a11y. reduce stays an expression rewrite upstream; the box-level reductions are a sibling pass, reduceBoxes(box, env).
• Contract test: a corpus of boxes (the existing tests/corpus.ts) must render without throwing in every library and match a display-list golden where the library uses layout.

Per-environment hooks and hints

A downstream library should say "on a phone, draw my CollectionTable as a list" without depending on any environment library. Three layers, in order of cost:

1. Hints as data on boxes (zero dependency): the option bag carries namespaced keys, { "native.Hint": "list", "tui.Width": 40 }; unknown keys are ignored (as Wolfram options). Carried through TagBox/StyleBox options — no new head. Good for sizes, orientation, Priority (what to drop first on a narrow screen).
2. A registry keyed by (head, environment), in the library's notation-style data (package.json entry plus a JSON file, like per-package notation, Speculative Overloads as Data): { head: "CollectionTable", native: { lower: "<boxes expression as markup>" }, tui: { lower: … } }. A hint's body is boxes or vdom markup, so it needs no environment code; the environment library reads the registry at mount. This is the default route.
3. A code hook, only where data cannot say it: the library exports an optional entry ./native (or ./tui) whose default export is (ctx) => HeadRenderer; the environment library import()s it when present. The library's package.json lists it under enumeratio.environments; absent means unhinted. Imports flow down the hierarchy: the hook imports boxes-ui types only, never boxes-nativescript.

Resolution order for one box: explicit option hint, registry row for this environment, code hook, the environment's own head renderer, reduction, text label.

Controls and options for every environment

A control is specified once, as data, in the vocabulary the reference already has (Slider(k, (0, 5)), PopupMenu(x, {…})):

• Required fields: name, kind, initial, domain (range/step or choices), label. These determine the pin and sample readings (Speculative Rendering Environments §5), so every control works on paper; nothing else may be required to be meaningful.
• Intent, not widget: a control declares what it chooses (continuous, choice, toggle, point2d, text), the environment picks the widget: slider/stepper/picker on a phone, a key-driven strip in the TTY, a menu in the web. ControlType stays a preference.
• Options are environment-keyed and ignorable: unknown options and native.* / tui.* keys are advisory, never errors; an option every environment honours has a plain name and gets a row in the control table before it ships. A new control or option PR adds its row for web, TUI, native and static, or says "inherits from X".
• Interaction contract (already controls.ts): binding in, ControlChange out, DOM-free. A native library maps ControlChange to its framework's state; gears (coarse/fine) become gestures (second finger, long press) per the Interaction axis.
• TUI improves in place: because the TUI draws the same boxes and the same control table, a better 2-D layout (via boxes-layout with cell metrics) improves every result at once.

Relation to the capability record

Environment stays the capability record and reduce the expression-level rewrite; this note adds the layer below it. Surface gains native, Interaction is unchanged (touch), form for native is the box default (no LaTeX string). A native environment is a record like COMPACT with engine: true, surface: ["native","vector"], layout: "compact"; the hooks key off environment.name. Open: whether a hint keys on a name (native) or on a predicate over the record (touch && compact) — a name is simpler and enough for now.

5. Spike plan (not executed)

Smallest proof: NativeScript-Vue renders a fraction and a slider-driven Manipulate from boxes on an iOS simulator, with no lit anywhere.

1. Run the engine (the risk first). A bare NS-Vue app imports @cortex-js/compute-engine, our boxes, and evaluates Binomial(5, 2) and one BigInt and one 50-digit N. Pass: it runs on V8 as shipped; note bundle size, cold start, whether WebAssembly exists. Repeat on RN/Hermes V1 only if NS fails.
2. Draw boxes as labels. A <BoxView> component walks RowBox/SuperscriptBox/TextBox into NS StackLayout/Labels (no real layout): proves the dispatch and the vdom mount. Pass: x² + 1.
3. Layout engine slice. layout() for FractionBox, scripts and SqrtBox with one bundled math font and measure via CoreText through NS; draw the display list as absolutely positioned labels plus rule views, and as SVG for comparison. Pass: a fraction, a binomial and a radical match MathML Core in Chromium at the same size (eyeball plus a display-list golden).
4. A Manipulate. ControlBox slider and DynamicBox readout from interfaceBoxes: the slider is NS Slider; changing it re-evaluates in the engine and re-lays out the box. Pass: a Manipulate(Binomial(n, k), (n, 0, 10), (k, 0, 5)) strip works at 60 fps by eye.
5. One hint. A downstream registry row that changes a CollectionTable to a list on native.
6. Decide: keep NS, switch to RN (Skia drawer), or fall back to Capacitor for mobile.

Cost: roughly a week for 1–2, two to three for 3–4; step 1 can end the effort in a day.

6. Answers so far (the maintainer, 2026-10-06)

• Feel or reach: both; one is better than none. Capacitor stays the reach baseline.
• NativeScript-Vue or React Native: the better fit, NativeScript-Vue. React Native pieces get ported as needed.
• Layout engine or SwiftMath/AndroidMath: the focus is layout, not math ergonomics. Drop in SwiftMath/AndroidMath for typeset math, as KaTeX is used for non-interactive rendering on the web, and spend the layout engine on everything else (§3). Math input is a separate question: rather than reimplementing MathLive, a MathQuill-style editor with MathLive's notation hooks is a possible direction later.
• Where the interface boxes live: open. It depends on how the split falls out; each environment may need its own base package, since kernel needs and capabilities differ by environment.
• Web components: implement a web component per box primitive, and build the more abstract heads (notatio-*) out of those smaller boxes. If that works, more generic implementations fall out of it.
• Hint keys: capabilities by preference; the exact environment kind where it matters (e.g. for choosing dependencies). So both: a predicate over the record, with the environment's name as one of its fields.
• BigInt: every expected host has a recent V8 or equivalent, so BigInt is assumed. Where a host lacks something, decline to interpret the affected symbols rather than answer wrong.

Sources

NativeScript 9.1 · Node-API engine previews · nativescript-vue · React Native 0.84 (Callstack) · Hermes BigInt #510 · react-native-svg · react-native-skia · Capacitor 8.5 · Lynx roadmap 2026 · Vue Lynx · SwiftMath · iosMath · Tauri 2 mobile survey