Add footnote syntax?

[chrisjsewell]

As discussed in #31 it appears that people are for a footnote syntax, as implemented in some flavours of Markdown (see Extended Syntax):

Here's a simple footnote,[^1] and here's a longer one.[^bignote]

[^1]: This is the first footnote.

[^bignote]: Here's one with multiple paragraphs and code.

    Indent paragraphs to include them in the footnote.

Can we come to a decision on this, bearing in mind

1. that this syntax extension is not strictly compliant with CommonMark, as discussed here
2. That (as with any syntax extension) this would incur some parsing performance cost (although probably minimal)

Note these footnotes would be inherently auto-numbered, as can be tried in the VS Code standard preview (GitHub markdown does not support footnotes)

paging @choldgraf @mmcky @jstac @akhmerov @najuzilu

[choldgraf]

On the question "should we have a syntax for footnotes?" I think the answer is "yes", since it's a common-enough syntax in writing documents.

To the question "which syntax should we use?" I think it should Pandoc syntax:

• [^footnotename] and later in the page: [^footnotename]: This is my footnote

This use-case is worth supporting since it is already quite similar to the rST footnote syntax.

The second question is whether to support inline footnotes, like this:

• ^[this is a footnote to be auto-numbered]

My intuition is that we should hold off on this one for now and revisit later on if it makes sense to accept this use-case (ie, if lots of people ask for it).

What do folks think?

[chrisjsewell]

Yep I’m for it. Just wanted to make clear the potential cons of its implementation.

[choldgraf]

What are your thoughts on inline footnotes? On the one hand, RMarkdown already supports this, but on the other hand, this would be breaking out of the strict "we don't do things that rST doesn't do" rules

[chrisjsewell]

What are your thoughts on inline footnotes? On the one hand, RMarkdown already supports this,

I'd say no because its pretty non-standard and could just as easily be handled in a simple extension:

{foot}`this is a footnote to be auto-numbered`

rather than introducing more complexity into the parser, leading to a slower parse and more potential bugs.

[choldgraf]

I like the idea of using roles for in-line footnotes, that's a clever idea. Gee who would have thought that extension points in a markup language would be useful 😅

[najuzilu]

I prefer Pandoc syntax - which is straightforward and simple - and I like @chrisjsewell's idea for in-line footnotes.

[akhmerov]

Would {^}`this is a footnote to be auto-numbered` be possible to handle using a role?

[choldgraf]

@akhmerov I think it would be possible but that this role doesn't yet exist, so somebody would need to write a little sphinx extension for it (similar to what Pandoc does...I believe that syntax is only enabled w/ an extension)

[akhmerov]

I meant to ask whether role names may contain special characters like ^ in myst. That would provide a compact syntax for inline references.

[choldgraf]

ohhh interesting - I'm not sure...maybe that's a question of whether rST allows for roles names with those characters?

[akhmerov]

But MyST's goal is not syntax compatibility, rather than AST compatibility, isn't it? That seems to be more within the responsibility of a parser.

[choldgraf]

yes-and-no, we want to be thoughtful and probably slow in deciding to branch off in new directions. One of the goals of MyST is "be as close to CommonMark as possible, but extend the syntax so we can use Sphinx". Adding in-line footnotes with special syntax would be another departure from CommonMark and in a direction that doesn't map on to a pre-existing Sphinx behavior. That's why I agree w/ @chrisjsewell 's intuition to use roles for this, and why I think the question is "would Sphinx allow for a role called ^?"

[akhmerov]

I think the question is "would Sphinx allow for a role called ^"

Seemingly the answer is "no"—the role names seemingly need to be alphanumeric.

[choldgraf]

Bummer - well, perhaps it could be ft or something quite short like that?

[chrisjsewell]

But MyST's goal is not syntax compatibility, rather than AST compatibility, isn't it? That seems to be more within the responsibility of a parser.

MyST's goal is to be spec compliant; this means parsing the tests outlined in: https://spec.commonmark.org/0.28/spec.json

As you can see here: https://github.com/ExecutableBookProject/MyST-Parser/blob/fb83d74243a6e8bf00e872ef4707c223d9fe56dd/tests/test_commonmark/test_commonmark.py#L17

All these tests currently pass, except a few minor non-compliances (reasons documented),
any change/addition to the syntax should have a very good reason for introducing any more non-compliances.

[chrisjsewell]

I think the question is "would Sphinx allow for a role called ^"

Seemingly the answer is "no"—the role names seemingly need to be alphanumeric.

This is the docutils regex for allowed role names:

    # Alphanumerics with isolated internal [-._+:] chars (i.e. not 2 together):
    simplename = r'(?:(?!_)\w)+(?:[-._+:](?:(?!_)\w)+)*'

So a no on ^

[choldgraf]

I think either way, we should table the "shorthand for inline footnotes" conversation to a different issue, and in the meantime can we agree that this style syntax:

• [^footnotename] and later in the page: [^footnotename]: This is my footnote

should be included in MyST?

[mmcky]

@choldgraf certainly need it supported and a short hand would be nice. We use it in lecture-source-py and any scientific publication needs that have footnotes supported.

I like the proposed syntax but I think @chrisjsewell is saying the regex excludes being able to use ^ as role :-(.

[choldgraf]

Correct me if I'm wrong, but I think he meant we can't use a directive or role with a special character in it, but we can still use it as part of the regex we search for w/ the myst parser.

[mmcky]

Correct me if I'm wrong, but I think he meant we can't use a directive or role with a special character in it, but we can still use it as part of the regex we search for w/ the myst parser.

oh I see. So for myst parser this is just a special syntax that would resolve [^] to {foot} role.

[choldgraf]

yeah, well it would probably just directly go from [^this syntax] to the docutils node for a footnote (at least, I think there's a special node for this). If we wanted to add an in-line role for a short-hand, it might be something like {ft}`This is my footnote text`

[chrisjsewell]

closed in #119