change markdown parser to mistletoe

[ghost]

Switched Markdown parser to Mistletoe.

Related to this issue.

[sirodoht]

Thanks for submitting this! It's great, just a couple of preferences changes

[sirodoht]

As far as the test failing, it seems mistletoe adds a new line by default to any single line renderings. To fix the test failing, I think you can change the last line of the main/tests/test_feeds.py file from:

f"<description>{self.user.blog_byline}</description>"

to:

f"<description>{self.user.blog_byline}\n</description>"

The change is an addition of a \n character in the content of the tag.

[ghost]

No problem! I'll do that when I get home, in a few hours.

[sirodoht]

I think I've made it so that you don't need my approval to run the CI anymore

[ghost]

All checks have passed now!

[sirodoht]

Great! I can’t recall if we talked about this but are the extensions we removed now supported natively? Namely tables, footnotes, toc.

[ghost]

Great! I can’t recall if we talked about this but are the extensions we removed now supported natively? Namely tables, footnotes, toc.

I don't know what toc is, but give me a few minutes and I'll test it!

Edit: It's "table of contents". Testing now...

[ghost]

Tables work, ToC doesn't, and neither do footnotes. Working on both now.

[ghost]

I want to get this working, but I don't know if I can figure it out. There isn't very detailed documentation for mistletoe, so it isn't the easiest thing to work with.

[sirodoht]

In terms of the TOC, I made an example here: https://nutcroft.com/blog/a-new-startup-lifecycle/

Essentially, the important part is that headings have ids. For example the first heading is an h2 and its id is i-the-philosophy-of-posthaven.

In other words it renders as <h2 id="i-the-philosophy-of-posthaven">I. The philosophy of Posthaven</h2>

For this reason, we can have a link such as https://nutcroft.com/blog/a-new-startup-lifecycle/#i-the-philosophy-of-posthaven which will scroll to that heading in the article.

The least important part, which we can do without, is the [TOC] marker, which generates these links in the beginning:

Let me know if these make sense!

[sirodoht]

In terms of footnotes, I see that mistletoe does not support them out of the box: miyuchina/mistletoe#47

One question is how easy we can re-implement them by overriding their HTMLRenderer, just like we did with the code highlighting part.

They don't have a footnotes example in their contrib folder as they do for code highlighting: https://github.com/miyuchina/mistletoe/tree/e853a0def73f4d4c2be5bf16ef4ef4135df0d556/mistletoe/contrib

But there is some kind of reference to footnotes in their docs building page here:
https://github.com/miyuchina/mistletoe/blob/e853a0def73f4d4c2be5bf16ef4ef4135df0d556/docs/__init__.py#L26-L34

as well as a Footnote BlockToken class, though I haven't figured out yet if that's the footnote we mean:
https://github.com/miyuchina/mistletoe/blob/e853a0def73f4d4c2be5bf16ef4ef4135df0d556/mistletoe/block_token.py#L745