raw_attribute not written by the markdown writer

[tolot27]

with pandoc 2.1.1 and a file header_with.md with content:

---
header-includes:
- '`\usepackage{todonotes}`{=latex}'
...

# Test

Some text.

and pandoc -s -t native header_with.md prints:

Pandoc (Meta {unMeta = fromList [("header-includes",MetaList [MetaInlines [RawInline (Format "latex") "\\usepackage{todonotes}"]])]})
[Header 1 ("test",[],[]) [Str "Test"]
,Para [Str "Some",Space,Str "text."]]

but the raw attribute is removed if pandoc -s -t markdown header_with.md gets called:

---
header-includes:
- '\usepackage{todonotes}'
---

\usepackage{todonotes}

Test
====

Some text.

It does not matter if I explicitly enable the raw_attribute extension for the input or output format.

Not only the raw attribute {=latex} gets removed, also the complete line \usepackage{todonotes} gets duplicated and printed after the yaml header.

[jgm]

The problem is that we can't tell from the AST whether you originally wrote `\foo`{=latex} or just \foo since both get parsed as a raw LaTeX element. The writer has to pick one rendering. One option would be always to pick the explicit raw attribute form if the raw_attribute extension is enabled. But this might give some unexpected results for a lot of people, especially since this attribute is set by default...

[tolot27]

I did not fully understand the rationale behind it. From the manual:

Raw content to include in the document’s header may be specified using header-includes; however, it is important to mark up this content as raw code for a particular output format, using the raw_attribute extension), or it will be interpreted as markdown. For example:

Hence, it becomes mandatory to mark up at least raw code in header-includes with the raw attribute. That is what I did.

Nevertheless, the header-includes should not be repeated outside of the yaml block. Even if it does not make much sense at all, pandoc -s -t markdown header_with.md -o - | pandoc -s -f markdown -t markdown repeatedly copies the header-includes statement outside of the yaml block.

[jgm]

Even if it does not make much sense at all, pandoc -s -t markdown header_with.md -o - | pandoc -s -f markdown -t markdown repeatedly copies the header-includes statement outside of the yaml block.

This is because the default markdown template includes a variable to be filled with the contents of header-includes. I agree, this gives weird results when your header-includes are already specified in the YAML metadata, but when we're resolving the template, we don't know whether header-includes were specified with an external file.

You can always remove this variable from your markdown template.

[jgm]

I think that using the explicit raw attribute for all raw content, if the extension is enabled, is the way to go.

[gpoore]

I just ran across a variation on this in pandoc 2.2. When round-tripping markdown (pandoc -f markdown -t markdown), it's currently possibly to use LaTeX macros to create markdown text (in which any LaTeX macros have been expanded).

Markdown example:

\newcommand{\code}[1]{`#1`{.lang}}

BEFORE \code{x} AFTER

BEFORE `` `x`{.lang}``{=latex} AFTER

Output of pandoc -f markdown -t markdown:

\newcommand{\code}[1]{`#1`{.lang}}
BEFORE `x`{.lang} AFTER

BEFORE `x`{.lang} AFTER

If this is then piped through pandoc -f markdown -t latex, we get

\newcommand{\code}[1]{`#1`{.lang}}

BEFORE \texttt{x} AFTER

BEFORE \texttt{x} AFTER

Meanwhile, sending the original markdown straight to LaTeX with pandoc -f markdown -t latex gives the very different output

\newcommand{\code}[1]{`#1`{.lang}}

BEFORE `x`{.lang} AFTER

BEFORE `x`{.lang} AFTER