Syntax to force md parsing inside latex environments

[memeplex]

Currently latex environments can't include markdown. There is the Begin/End trick and the native div workaround. Both have similar shortcomings: it's not clear how to define an environment with intermixed content (see my next comment). Since it's not possible to write a reentrant filter I'm proposing to add a syntax to force parsing of markdown. The upcoming div syntax could provide the base syntax (inline and block) and be special cased for some specific class, say :::md.

[memeplex]

Just to be clear: when referring to mixed content above I was thinking about environments like a table where text is intermingled with special syntax in a way that is impossible to disentangle by the pandoc reader. I don't have enough experience with complex latex documents to assess the practical relevance of those use cases, though.

[memeplex]

Another option, although tex specific, would be to have a distinguished "tex command", say \md{}, meaning that everything inside it will be parsed. The name might be changed by a command line option to avoid clashes. Since it's tex syntax mixing tex and markdown would be business as usual.

[jgm]

You're talking about latex environments in markdown
documents, or in latex documents? (Are we talking
pandoc -f markdown or pandoc -f latex?)

+++ memeplex [Oct 05 16 00:46 ]:

Currently latex environments can't include markdown. There is the
Begin/End trick and the native div workaround. The first is all or
nothing: if some environment contains latex it can't contain markdown
also. The second would be improved with the upcoming(?) div syntax but
it has a similar shortcoming: it's not clear how to define a mixed
environment. Since it's not possible to write a reentrant filter I'm
proposing to add a syntax (say md ..., maybe both inline and block...
whatever) to force parsing of markdown.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, [1]view it on GitHub, or [2]mute the
thread.

References

1. Syntax to force md parsing inside latex environments #3145
2. https://github.com/notifications/unsubscribe-auth/AAAL5MDT7yImDxjr6hrUQsZzQZ6l5axDks5qw1XlgaJpZM4KOggR

[memeplex]

I'm mainly interested in the markdown reader. Maybe there is some value in supporting the tex reader also but I'm not able to see how someone that already embraced tex syntax would be significatively benefited by inlining small fragments of md.

[ickc]

@memeplex

You can try the links in https://github.com/jgm/pandoc/wiki/Pandoc-Filters under "LaTeX related" section. And also the "pandoc-amsthm" link there. I just read a trick in Markdown inside LaTeX code - Google Groups too.

[memeplex]

@ickc are you referring to latexdivs? I know the native div trick but the syntax is clumsy and nesting is very limited. That's why I'm proposing to leverage tex syntax for this (the same than html syntax is used for nesting markdown in html elements).

[ickc]

Yes. I remember someone opened a respository planning to get markdown in LaTeX syntax in pandoc (I can't find the link now). But it seems he's hitting a brick wall and go back to the idea of using pandoc div and a filter to transform it into LaTeX environment.

I remember there are some open issues here concerning the pandoc LaTeX parser (I mean, raw LaTeX in markdown) that has some bugs. In a sense the problem is really how to tell something is raw LaTeX or not. Allowing markdown in LaTeX will further complicate the issue, given it is already not perfect.

I'm not saying it is not impossible to have a syntax to force markdown rendering. But I guess such a syntax has to be elegant enough in order to make into the official pandoc. e.g. the official markdown-ish syntax for div and span has already been discussed for more than at least 5 years (just in GitHub issue tracker, not to mention any pandoc discuss before that). So if you are interested in any of the syntaxes you proposed, you'd better wrote a filter and get that working now rather than waiting a few years.

I guess the reason that markdown in HTML is much easier to implement is because the HTML syntax (the use of <>). LaTeX's syntax using \{} etc. are all used in pandoc's markdown syntax. So it seems to be very easy to mingle markdown and LaTeX while hard to tell which is which.

Lastly, I actually don't understand the problem you mention about the shortcoming of the using the pandoc div and how they are intermingled. Can you provide a concrete MWE, using either one of those filters that convert div to LaTeX environments?

[bpj]

@memeplex, there is a fairly easy LaTeX hack to hide environments from Pandoc. Put this in your LaTeX template or a -H file:

\let\Begin\begin
\let\End\end

To LaTeX \Begin and \End will now be synonyms of the ordinary lowercase versions, so you can put \Begin{foo} and \End{foo} in your Markdown file and LaTeX will see an environment but Pandoc will see them as ordinary commands and parse Markdown between them. You may need to put blank lines before and after them though.

[cboettig]

@bpj This is a clever workaround! Unfortunately, it's not clear to me how to apply it in the case where the \begin takes optional arguments, e.g. \Begin{tcolorbox}[title= Box 1: Machine learning under tipping points, lower separated=false]. Any suggestions?

[ickc]

I think it is just the same, (i.e. `\begin{....`.) Did it fail you? Yours, Kolen

…

On Jul 3, 2018, 12:37 PM -0700, Carl Boettiger ***@***.***>, wrote: @bpj This is a clever workaround! Unfortunately, it's not clear to me how to apply it in the case where the \begin takes optional arguments, e.g. \Begin{tcolorbox}[title= Box 1: Machine learning under tipping points, lower separated=false]. Any suggestions? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cboettig]

@ickc thanks, this does create the environment, only the options in the square brackets are not applied to the environment, but instead print literally into the output (i.e. literal [title= Box 1: My title, lower separated=false] at the beginning of the box). (testing in Pandoc v2.1.1)

[bpj]

It happens because Pandoc sees \Begin as an ordinary inline command, and doesn't understand unorthodox command signatures like arguments in square brackets after arguments in braces, or arbitrary delimiters as supported by the xparse package. The workaround is to define a dummy command which just "returns" to its only argument and hide the unorthodox command from Pandoc by wrapping it in the dummy command.

Near the top of the Markdown document or in a header include say:

\newcommand\dummy[1]{#1}

and then wherever you have an unorthodox command, or in this case \Begin:

\dummy{\Begin{someenv}[optional]}

It is important not to create a group inside the definition of the dummy command.

[cboettig]

@bpj Very clever, that works perfectly 🎉 . Many thanks!

[sinisterstumble]

The extension raw_attribute solves this. (ignore the >)

> ```{=latex}
> \begin{center}
> ```
> Remember to check **Appendix A** for additional resources references and detailed explanations.
> ```{=latex}
> \end{center}
> ```

becomes

\begin{note}
Remember to check \textbf{Appendix A} for additional resources references and detailed explanations.
\end{note}

[jgm]

Raw attribute is the way to go. Closing this.