prefer marked to pandoc for markdown2html

[minrk]

If node is installed, the exact same code will be used to render markdown2html as the live notebook.
Installing marked with npm is not required.

[damianavila]

via marked...

[Carreau]

1 comment, still, we used to use python-markdown and removed it for uniformly using pandoc. Not that I am opposed to using marked. But it feel like a step forward for a step back.

[ahmadia]

@Carreau - The difference here is that Python Markdown was independent of the parser that was used to present web pages. I think this might be a step sideways.

[minrk]

But it feel[s] like a step forward for a step back.

It certainly is. We have a few choices:

1. use python-markdown for HTML output
   • + Python dependency
   • - doesn't support the same markdown features (gfm, tables, etc.) as live or pandoc
2. use pandoc
   • + same reader for markdown all over nbconvert
   • + supports all the same extended markdown as marked (gfm, tables, etc.), and more
   • - but it still parses things a bit differently, especially wrt math / latex, empty lines
3. use marked
   • + same renderer as live notebook
   • - have to extract/re-add math around it, like we do in js (incorrectly, atm)
   • - depends on node

Honestly, none of these are awesome. In some respects, bringing pandoc up to snuff via either plugins or patches is the most appealing. I think python-markdown is a no-go, with too many unsupported behaviors, unless we want to turn off the extensions in marked.

[ahmadia]

• doesn't support the same markdown features (gfm, tables, etc.) as live or pandoc

I think it supports these via extensions. See this post for an explanation of how to get GFM. I think I've seen something similar for getting MathJax working as well. My biggest concern is that two different code paths will ultimately be untenable, and that we'll either need Python in the browser via a tool like Native Client, or we'll have to live with more Javascript at the command line. Pandoc is an awesome tool, but I don't see it in the browser any time soon.

[takluyver]

There's a list of extensions for python-markdown here. It includes tables and github style code blocks. Does that get us close to what's supported in the browser?

There are also third party extensions, which include some math/latex tools.

[minrk]

• We cannot avoid using pandoc (for outputs other than HTML)
• We cannot avoid using marked (browser-side)

So I am not thrilled about the idea of have at least three markdown implementations to contend with, but it's worth considering if the right extensions can be brought to bear. Right now, I would consider python-markdown the least desirable of the three options.

[damianavila]

three markdown implementations

😳

if the right extensions can be brought to bear

and with extensions... 😒

Maybe to much cost?

[ellisonbg]

Just based on principle, I would rather use the same Markdown parser in both the live notebook and nbconvert, even if there are things it doesn't do correctly. Installing node is not too much more painful than pandoc - but we should definitely fall back on pandoc. All that to say, I am probably +1 on using marked in nbconvert. Extra bonus points if we could figure out how to call our own math add/remove code from node for nbconvert.

[minrk]

This now should use the full code, including mathjaxutils, that we use in the live notebook. Also, it uses our marked, not node's, so the only dependency should be node itself.

[ahmadia]

Awesome. Let me know if I can help verify/test this.

[damianavila]

Travis is 😢, jeje...

[minrk]

tests passing

[minrk]

Merging soon, unless objections.

[damianavila]

OK, I read it again... it seems ready for me 👍