Post: MathJax 3

_posts/2020-03-29-mathjax3.md

@@ -0,0 +1,103 @@
+---
+title: Painful Upgrade to MathJax 3
+layout: post
+use_toc: true
+use_code: true
+excerpt: The upgrade experience to MathJax 3 was far from smooth.
+---
+
+
+## Writing Math in Markdown
+
+### Jekyll kramdown
+
+Jekyll uses the [kramdown](https://kramdown.gettalong.org/) converter to compile Markdown into HTML. As part of the conversion process, kramdown looks for ["math blocks"](https://kramdown.gettalong.org/syntax.html#math-blocks) that start and end with `$$`. The double-dollar-sign is the only math block delimiter that kramdown supports. A math block that is both preceded and follwed by a newline is considered "display math." Any non-display math block is called "inline math."
+
+There is one caveat: `$$`-enclosed text that appears between an explicit pair of opening and closing HTML tags in the Markdown file might need special treatment. See the [kramdown documentation](https://kramdown.gettalong.org/syntax.html#html-blocks) on `markdown="span"` and `markdown="block"` attributes.
+
+The [`math_engine` option](https://kramdown.gettalong.org/options.html) in Jekyll's `_config.yml` determines what kramdown does with each math block. The default is `math_engine: mathjax`, which converts the math block double-dollar-sign delimiters into `<script>` tags ([kramdown documentation](https://kramdown.gettalong.org/math_engine/mathjax.html), as follows:
+- display math: `[newline] $$...$$ [newline]` converted to `<script type="math/tex; mode=display">...</script>`
+- inline math: `[text] $$...$$ [text]` converted to `<script type="math/tex">...</script>`
+
+This behavior is appropriate for MathJax 2.7, which renders the output `<script>` tags into math notation. However, MathJax 3 no longer uses the `<script>` tags, so kramdown's math block processing is no longer necessary and should be disabled by `math_engine: nil`. This is explained in more detail below.
+
+Another important note is that kramdown treats the `\` character as an escape character. In order to have `\(` or `\[` appear in the generated HTML output, the Markdown file must have `\\(` or `\\[`.
+
+
+### MathJax
+
+MathJax is a JavaScript library that parses HTML documents, identifies math expressions, and renders them in proper math notation. In both versions 2.7 and 3, MathJax looks for certain math delimiters:
+
+> The default math delimiters are `$$...$$` and `\[...\]` for displayed mathematics, and \(...\) for in-line mathematics. Note in particular that the `$...$` in-line delimiters are **not** used by default.
+
+*See MathJax [2.7](https://docs.mathjax.org/en/v2.7-latest/start.html#tex-and-latex-input) and [3](https://docs.mathjax.org/en/v3.0-latest/basic/mathematics.html#tex-and-latex-input) documentation*
+
+Note that these delimiters are customizable. See MathJax [2.7](https://docs.mathjax.org/en/v2.7-latest/tex.html#tex-and-latex-math-delimiters) and [3](https://docs.mathjax.org/en/v3.0-latest/input/tex/delimiters.html).
+
+In version 2.7, MathJax ran a [preprocessor](https://docs.mathjax.org/en/v2.7-latest/advanced/model.html) on HTML files that converted math delimiters into `<script>` tags, similar to the process described for kramdown above. Then, a MathJax "input jax" would translate each `<script>` tag into the internal MathJax format which is then rendered by an "output jax" based on the output format chosen by the user (e.g. HTML-CSS, SVG, or NativeMML).
+
+However, MathJax 3 changes the paradigm by skipping `<script>` tags altogether. Instead, it directly parses math delimiters in HTML documents and renders math expressions. As a result, MathJax 3 by default no longer accepts the `<script>` tags that kramdown outputs using `math_engine: mathjax` option; instead, kramdown's math block processing should be disabled by setting `math_engine: nil` in order to function properly with MathJax 3. As a [workaround](https://docs.mathjax.org/en/latest/upgrading/v2.html#changes-in-the-mathjax-api), MathJax 3 can also be extended to work with HTML documents that already have `<script>` tags by adding an action to its `renderActions` option. The first element in the list corresponding to the `findScript` key is a number indicating the priority that the action runs. For reference, MathJax 3's standard parsing is done using the `find` action with a priority of 10.
+
+```javascript
+window.MathJax = {
+  options: {
+    renderActions: {
+      findScript: [9, function (doc) {
+        for (const node of document.querySelectorAll('script[type^="math/tex"]')) {
+          const display = !!node.type.match(/; *mode=display/);
+          const math = new doc.options.MathItem(node.textContent, doc.inputJax[0], display);
+          const text = document.createTextNode('');
+          node.parentNode.replaceChild(text, node);
+          math.start = {node: text, delim: '', n: 0};
+          math.end = {node: text, delim: '', n: 0};
+          doc.math.push(math);
+        }
+        doc.findMath();
+      }, '']
+    }
+  }
+};
+```
+
+Other sources
+- [MathJax 3 in Jekyll and Kramdown](https://11011110.github.io/blog/2019/10/17/mathjax-3-jekyll.html): describes the kramdown-MathJax 3 integration problem and proposes a different JavaScript solution to deal with the `<script>` tags that kramdown generates.
+
+
+### GitHub Pages
+
+As described above, it is preferable to set `math_engine: nil` in Jekyll's `_config.yml` when using MathJax 3. However, GitHub Pages currently always sets `math_engine: mathjax` in `_config.yml`, overriding any user-specified value ([GitHub Help](https://help.github.com/en/articles/about-github-pages-and-jekyll)). This means that all math delimited by `$$...$$` are replaced with `<script>` tags. This is a GitHub Pages limitation, and not a Jekyll limitation.
+
+
+## Current Setup
+
+I have described how kramdown, MathJax, and GitHub Pages work individually. Now I describe what it means when these moving pieces all come together.
+
+Because I am choosing to upgrade to MathJax 3 and I am using GitHub Pages to automatically generate my static site, I am stuck with Jekyll's `math_engine: mathjax` option in `_config.yml`. Therefore, math blocks `$$` in Markdown are converted to `<script>` tags, which I parse with MathJax by adding in the `findScript` action to its set of `renderActions`.
+
+I could have tried to avoid kramdown's math blocks altogether by switching to single-dollar-sign `$` delimiters for inline math and using `\[...\]` delimiters for display math. However, this strategy does not always work kramdown does not know to treat the math expression as a math block and may instead interpret LaTeX underscores as italics. Furthermore, I am also holding out for the [possibility](https://github.com/github/pages-gem/pull/644) that GitHub removes its `math_engine: mathjax` requirement.
+
+Thus, my current setup is as follows:
+
+**Setup**
+- Set `use_math: true` in the preamble of the page. To write an actual dollar-sign in the page, escape it as `\$`. Such escaping is only necessary when `use_math: true` is set in the preamble.
+
+**Inline Math**
+- _legacy (but current) option_: `[text] $$...$$ [text]` (note the double dollar signs)
+    - converted by `kramdown` into `[text] <script type="math/tex">...</script> [text]`
+    - MathJax uses the `findScript` action in `renderActions` of the MathJax configuration to identify these script tags
+- _future (hopefully)_: `[text] $...$ [text]` or `[text] \\(...\\) [text]` (double-backslash is necessary because `kramdown` treats `\` as an escape character)
+    - not converted by `kramdown` during `jekyll` Markdown-to-HTML compilation
+    - rendered directly from LaTeX by MathJax 3 (after enabling `$` inline delimiter)
+
+**Display Math**: `[newline] $$...$$ [newline]`
+- _legacy (but current) option_
+    - converted by `kramdown` into `<script type="math/tex; mode=display">...</script>`
+    - MathJax 3 uses the `findScript` action in `renderActions` of the MathJax configuration to identify these script tags
+- _future (hopefully)_
+    - not converted by `kramdown` during `jekyll` Markdown-to-HTML compilation
+    - rendered directly from LaTeX by MathJax 3
+
+
+## Known Issue
+
+Because the `findScript` action is called separately from `find`, different math expressions are parsed at different times. Suppose the `findScript` action is called before `find`, as illustrated in the example above. As a result, if there are new commands or operators (*e.g.* `\newcommand{\bx}{\mathbf{x}}`) that are used by a math expression parsed by `findScript` but defined within an expression parsed by `find`, then the expression will fail to display properly.