Markdown reader doesn't handle hard breaks

[alerque]

Markdown has a syntax for specifying hard line breaks by ending the line with a \. This is ignored by the internal markdown reader and they come out in the output. (See also #413.)

Additionally there is a Pandoc flavored extension to markdown that begins poetry formatted blocks (i.e. respect line breaks and initial spaces) with a |. That format is likewise ignored.

[severak]

I didn't know about that poetry formatted blocks before. 👍

It's a pity that these useful tricks from pandoc are not portable to other markdown dialects and causes glitches, e.g. this leaking backshlashes at GitHub,

[alerque]

@severak I'm pretty sure your use of backslashes in that particular case is actually wrong. You're combining a two-space thing on empty lines with a backslash and I don't think they make any sense together. Pandoc figures out to do something, but it's document logic is also alien magic compared to most. As long as you're going with something only Pandoc can parse properly it would bake a lot more sense to use the block level syntax:

# Title

| Lines of
| Poetry here
|
| Second stanza
| Over the rainbow

[Omikhleia]

There's a bunch of different things here:

• Hard line breaks
  • Markdown does have a standard syntax: two trailing spaces at the end of a line.
    • This syntax is actually supported by lunamark (the parsing Library used in our internal native markdown support)... but it wasn't properly handled (mapped) in our custom internal reader 👀
    • markdown.sile now correctly handles it.
  • Pandoc-extended Markdown has a alternative syntax - i.e. an extension - for, indeed "specifying hard line breaks by ending the line with a \". This is the escaped_line_breaks extension, which Pandoc enables by default).
    • It wasn't supported by lunamark, but my PR just got accepted... Yay!
    • ... and obviously my markdown.sile handles it too.
• Line blocks (Pandoc's line_blocks extension), which are indeed interesting for typesetting addresses, etc. ... and poetry!
  • These weren't supported by lunamark, but again, my PR for them just got accepted too... Yay!
  • ... and again markdown.sile leverages them, with extra provisions for poetry in mind.

In other terms, it seems to me that this 4-year old issue could be closed.

I'll comment later at some point in #1336 regarding the whole progress of the markdown.sile package (I still have a few things under the hood), but as far as line breaks and line blocks go, for fairly good parity with Pandoc, this is it!

(As can be seen there, markdown.sile also includes, besides the "native" rendering engine, a way to use Pandoc's JSON AST, where obviously all these features are also supported too)

[alerque]

Closed via #1610.