Latex not parsed correctly by markdown renderer in VS Code

[DonJayamanne]

Discussed in microsoft/vscode-jupyter#9630

^{Originally posted by jhconning April 7, 2022}
The Quarto notebook project builds part of its documentation around how to use vs-code with the jupyter extension to create beautifully rendered HTML and pdf documents https://quarto.org/docs/get-started/hello/vscode.html

However, I've noticed a problem with using Quarto syntax that results in a katex parsing error in a markdown cell when using the juptyer extension. The issue does not arise using JupyterLab.

Quarto allows one to label and cross-reference equations, as described here:
https://quarto.org/docs/get-started/authoring/vscode.html#cross-references

In the example there, a {#eq-stddev} label is added immediately after the $$ delimiter (does not work if the label is on the next line). The rendered quarto document then produces a numbered equation and the @eq-stddev label can be used elsewhere in the document to create clickable cross-reference links.

## Equation 
THis is the standard deviation formula:
$$
\sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \overline{x})^2}
$$ {#eq-stddev}

The problem is that the juptyer extension in vs-code chokes on this syntax, and presents a Katex parser error instead of the rendered cell:

This problem does not arise in Jupyterlab, where the same is rendered like this:

Would it be possible to change the parsing to allow this syntax? One could then develop notebooks using the jupyter extension and also have the option of using the powerful extra directives and syntax from quarto to produce beautiful websites, books, blog posts, and other documents.

[mjbvz]

Looks like the parser doesn't like the trailing {#eq-stddev}. Based on the screen shot it looks like Jupyter just ignores this text

Here's the relevant code to look into fixing this: https://github.com/mjbvz/markdown-it-katex

[Lemmingh]

Jupyter just ignores this text

Jupyter can easily make it because its Markdown parsing is powered by Marked, as I noted in #141905 (comment) and yzhang-gh/vscode-markdown#1030 (comment).

Marked is not CommonMark-compliant. Indeed, they disagree at the very first step of parsing (block-level): markedjs/marked#1202

---

{#eq-stddev}

This seems a derivative of "pandoc-eqnos". To allow it in markdown-it-katex, the end condition of math_block perhaps needs to be changed to something much more complex.

---

Well, I'd recommend just replicating Jupyter's Marked for processing Markdown in a Jupyter notebook. This could potentially solve half of the problems occurring in rendering Jupyter Markdown by markdown-it.

[Lemmingh]

Since Jupyter does not allow blank line inside an equation, there is still a chance of mimicking its logic in a markdown-it plugin:

Only contribute inline rules. In other words, all math areas must be treated as inline structures at Markdown level.

This is not a rare choice, but against the design of markdown-it-katex.

---

From this perspective, I recommend building a separate dedicated markdown-it plugin. Or you can choose to make your fork mostly incompatible with the original one.

markdown-it-katex now defines several inline and leaf block structures.

So, the first thing is to merge them into one inline rule. What about simply naming it "math", or verbosely "math_span"? Anyway, this step is complicated.

The remaining work is just adjusting rendering calls (renderer rule). We can distinguish structures by Token.markup as long as it's properly set during parsing.

[JerryWJi]

I'd like to take a stab at this - not completely sure where to start though, would appreciate some hints or leads!

[mjbvz]

@JerryWJi The relevant plugin is here: https://github.com/mjbvz/markdown-it-katex

Once you fix the issue in the repo — which includes tests you can use to verify the fix — we can then pick up the fix in VS Code

[JerryWJi]

@mjbvz Thanks for the help. The approach I'm currently taking involves changing this line here: https://github.com/JerryWJi/markdown-it-katex/blob/master/index.js#L176

Am I going down the right path, or is there something else I should try?

[JerryWJi]

@mjbvz pushed a code fix to my fork - the code no longer only checks the last two characters for the terminating double dollar sign. https://github.com/JerryWJi/markdown-it-katex/blob/master/index.js#L181

Let me know if this is looking good so far.

[mjbvz]

@JerryWJi Can yon please try creating a PR for this so I can more easily review it? Also be sure to add some tests to confirm the new behavior