Switch to a CommonMark compliant parser

[kidonng]

Marked is not CommonMark compliant. It has improved significantly since I last looked at it, but there are still a few edge cases.

I suggest switching to a CommonMark compliant parser if deno-gfm is aiming for feature parity with GitHub.

There are a few choices:

• remark: the most obvious choice. For starters, remark-gfm solves Add support for footnotes #6 with zero efforts.
• markdown-it: maintenance seems to be stalled_{, and I cannot fathom why VitePress chooses it if it's starting anew}.
• marked: believe it is already good enough and will eventually be compliant

I won't dive deep here because remark people has written awesome comparisons.

Personally I'm strongly in favor of remark, it makes future development easier than the other choices.

[lino-levan]

I think that if we move to anything, remark-gfm seems to make the most sense. We should think about how this change would interact with #55. Are there any regressions that we would expect switching to remark-gfm (outside of it being less battle-tested?)

[kidonng]

We should think about how this change would interact with #55.

This would/should go before any other changes I proposed, as I intended it to be.

Are there any regressions that we would expect switching to remark-gfm (outside of it being less battle-tested?)

It is battle-tested. Any regression would just be mismatching current code.

[lino-levan]

This seems like a good step forward, especially if we get footnote support "for free" along with the change. Are you interested in making this PR or should I do it? This should cleanup our codebase quite a bit given that remark has the remark-gfm plugin as well as remark-math.

This would also help close these at deno_blog

• feature request: footnotes deno_blog#121
• Add Math Rendering deno_blog#118

[lino-levan]

Could also close #51

[kidonng]

It seems I'm less familiar with the current state of this/related projects than you, I would appreciate it if you are going for it.

[hashrock]

I'm interested in it. Does remark work without npm specifier in deno? Has anyone tried it?

[lino-levan]

It seems to just work out of the box with no crazy configuration needed. This results in:

<p><em>hello there</em></p>

[hashrock]

Looks great!

However, there may be an option to use it directly without a wrapper. Especially when customizability is important, such as dotland/x

[lino-levan]

I was playing around with remark and its family of plugins and there are some subtle issues that make it pretty hard to switch easily. The biggest one is there is somehow no syntax highlighting plugin that uses prism (that doesn't use node:vm) and we can't really switch to alternatives (since it looks like #67 has stalled). We could write this ourselves.

[lino-levan]

@hashrock I see you've been playing around with remark. Any luck on syntax highlighting?

[hashrock]

@lino-levan
I haven't tried prism, but rehype-highlight (lowlight) worked. (starrynight didn't work though)

I'd like to see the impact on speed, bundle size, and rendering fidelity of GitHub's README markdown.
https://github.com/hashrock/simulate-github-gfm

The rendering of katex is a little strange at the moment. I think maybe I'm using it wrong.

[lino-levan]

If I recall correctly, I was getting a similar issue last I checked. Would moving to lowlight be fine? I don't know how important syntax highlighting is outside of it existing.

[hashrock]

Hmm, I'm not sure yet. I think there is no choice but to try rendering the actual GitHub README and accumulate evidence.

Another way is to create a separate library that wraps remark. It allows things to move faster and allows projects like deno_blog to choose libraries at their own discretion.

[hashrock]

Study

A comparison of remark and marked can be found here:

https://github.com/micromark/micromark#comparison

If you have markdown you trust and want to turn it into HTML without a fuss, and don’t care about perfect compatibility with CommonMark or GFM, but do appreciate a small bundle size and stability, use marked

so I checked the bundle size with a script that has roughly the same functionality.

https://github.com/hashrock/simulate-github-gfm/blob/main/simple-remark.ts

% deno info simple-marked.ts
dependencies: 64 unique
size: 1.37MB

% deno info simple-remark.ts
dependencies: 252 unique
size: 1.4MB

There isn't much difference in size, but the number of dependencies has increased significantly. I'm thinking that it may affect the initial cache time and deployment time, but the time measured by deno cache -r varies considerably and I couldn't measure it well.

[lino-levan]

Interesting. Thanks for the notes!