Multiline comments

[hukkin]

Is your feature request related to a problem? Please describe.

This is a proposal to change parsing of MyST comments.

Example Markdown:

% This is the first line of a comment, but it's a long comment so
% here's the second line.

MyST comments are currently parsed so that each line is its own token. Looking at the example Markdown above, it would feel natural to me if the comment was one continuous block however.

This would be in line with how CommonMark block quotes and code blocks are parsed:

> This is one multiline
> block quote!

    This is one multiline
    indented code block!

This change would:

1. Make the syntax structure more intuitive and in line with existing CommonMark.
2. Simplify rendering and enable word wrapping in renderers such as mdformat-myst.

Describe the solution you'd like

Parse a token similar to fence. That is, a single token (no openers or closers), and have all the multiline content in token.content.

[chrisjsewell]

Sounds potentially reasonable (would have to double check first).
Well here's the markdown-it-py plugin: https://github.com/executablebooks/mdit-py-plugins/blob/b1221ffc5224c8e713aac1f91f3ef1e4abe30c46/mdit_py_plugins/myst_blocks/index.py#L34, and you know how to write these things, so feel free to try out a PR 😉

(I'm also planning to revisit: https://github.com/executablebooks/mdit-py-plugins/blob/b1221ffc5224c8e713aac1f91f3ef1e4abe30c46/mdit_py_plugins/myst_role/index.py#L16, to make it multiline (#269))

[hukkin]

If I do end up making a PR, another thing I'd like to change is to not strip line leading whitespace. This is because comments (at least in proper coding languages) sometimes do stuff like

#              ⬇ This is the second arg
def func(arg1, arg2):
    pass

or have ASCII art or whatever in them. Stripping leading whitespace makes these things unrecoverable.

What do you think?