Pandoc versions 2.13+ include the yaml header in the output for GFM

[jimhester]

With pandoc 2.12 this works as expected.

tmp <- fs::dir_create(fs::file_temp())
reprex::reprex(input = "rmarkdown::pandoc_version()\n", venue = "gh", wd = tmp, advertise = FALSE)
readLines(fs::dir_ls(tmp, glob = "*md"))
#> [1] "``` r"                       "rmarkdown::pandoc_version()"
#> [3] "#> [1] '2.12'"               "```"

With pandoc 2.13 we get the YAML header as well

tmp <- fs::dir_create(fs::file_temp())
reprex::reprex(input = "rmarkdown::pandoc_version()\n", venue = "gh", wd = tmp, advertise = FALSE)
readLines(fs::dir_ls(tmp, glob = "*md"))
#>  [1] "---"                          "author: jhester"             
#>  [3] "date: 2021-04-28"             "output:"                     
#>  [5] "  reprex::reprex_document:"   "    advertise: false"        
#>  [7] "title: legal-mink\\_reprex.R" "---"                         
#>  [9] ""                             "``` r"                       
#> [11] "rmarkdown::pandoc_version()"  "#> [1] '2.13'"               
#> [13] "```"

^{Created on 2021-04-28 by the reprex package (v2.0.0)}

It looks like this is fallout from jgm/pandoc#6537, pandoc now parses the YAML header for GFM, before it was silently discarded.

I think likely reprex will have to either remove the header from the input file it sends to pandoc, or remove it from the output after the fact.

[jennybc]

Cc @cderv

Is this going to have effects beyond reprex in the knitr-verse and I can maybe rely on it getting "fixed" in knitr? (Caveat: this is a quick reaction to this issue, not based on actual exploration yet.)

[cderv]

Yes I think we need to cover this new case in md_document() which has preserve_yaml=FALSE by default.

We don't have issue with github_document() because the output template does not include title block. But we need to change that also if we want this former fonction to support preserving yaml header.

Thanks for the ping.

[jennybc]

Sorry if it should be clear but ... do you think I will need to deal with this explicitly in reprex then?

[cderv]

To answer that it required a deeper look.

The bottom issue is because --standalone pandoc arg is passed in the command line. That is what makes the yaml header added with this new yaml_metadata_block extension activated by default.

So on rmarkdown side I think the correct fix is that rmarkdown::md_document("gfm", preserve_yaml = FALSE) should remove the new extension yaml_metadata_block. Currently md_document() is assuming Pandoc does not add yaml block except for markdown format.

However, on reprex side, I think you will need to deal with it even with a fix in rmarkdown because you are not using rmarkdown::github_document() or rmarkdown::md_document("gfm"). You are using default md_document() as based format

reprex/R/reprex_document.R

Lines 113 to 117 in d6b12e4

	format <- rmarkdown::output_format(
	knitr = rmarkdown::knitr_options(
	opts_knit = opts_knit,
	opts_chunk = opts_chunk
	),

This means that you inherit the argument for Pandoc for the default behavior of md_document() which is equivalent to md_document(variant = "markdown_strict") and this insert the --standalone arg you don't want. That is what is causing the issue.

In the first place, I think reprex should use rmarkdown::md_document("gfm") as base format, and with a fix in rmarkdown this should work as expected.

If you want a workaround in the meantime without waiting for a new rmarkdown version, I think you can deactivate the extension in your format directly and that would work

diff --git a/R/reprex_document.R b/R/reprex_document.R
index 9d5a991..16740c5 100644
--- a/R/reprex_document.R
+++ b/R/reprex_document.R
@@ -116,7 +116,7 @@ reprex_document <- function(venue = c("gh", "r", "rtf", "html", "slack", "so", "
       opts_chunk = opts_chunk
     ),
     pandoc = rmarkdown::pandoc_options(
-      to = "gfm",
+      to = paste0("gfm", if (rmarkdown::pandoc_available("2.13")) "-yaml_metadata_block"),
       from = rmarkdown::from_rmarkdown(implicit_figures = FALSE),
       ext = ".md",
       args = pandoc_args