number-sections does not number headers with GFM output format

[ivan-paleo]

Bug description

When rendering to GFM, number-sections: true does not add numbers to headers, only to the ToC.
It works as expected when rendering to HTML. I could not find an existing issue or a discussion in the Q&A, but maybe I've missed it.
Since number-sections is listed in the GFM options, I would expect it to work just like with HTML. If not possible, maybe the help should be adjusted.

Steps to reproduce

---
title: "test"
format: 
  gfm: default
toc: true
number-sections: true
---

# header1

# header2

## subheader2-1

## subheader2-2

Expected behavior

Same as with HTML:

---
title: "test"
format: 
  html: default
toc: true
number-sections: true
---

# header1

# header2

## subheader2-1

## subheader2-2

Actual behavior

The ToC is numbered but not the headers when rendering to GFM. Both are numbered when rendering to HTML.

Your environment

VScodium Version: 1.85.1 (user setup)
Release: 23348
Commit: 08e6c15293922dd53a864bb041be381322fee401
Date: 2023-12-14T20:44:42.949Z
Electron: 25.9.7
ElectronBuildId: undefined
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045

Quarto check output

[>] Checking versions of quarto binary dependencies...
Pandoc version 3.1.1: OK
Dart Sass version 1.55.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
Version: 1.3.450
Path: C:\Program Files\Quarto\bin
CodePage: 1252

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
Version: 3.8.5
Path: C:/Users/calandra.RGZM/AppData/Local/Programs/Python/Python38/python
.exe
Jupyter: (None)

  Jupyter is not available in this Python installation.
  Install with py -m pip install jupyter

[>] Checking R installation...........OK
Version: 4.3.2
Path: C:/PROGRA1/R/R-431.2
LibPaths:
- C:/Users/calandra.RGZM/AppData/Local/R/win-library/4.3
- C:/Program Files/R/R-4.3.2/library
knitr: 1.45
rmarkdown: 2.25

[>] Checking Knitr engine render......OK

[mcanouil]

I cannot reproduce using 1.4 pre-release.
Could you try with 1.4 pre-release?

[ivan-paleo]

The sections are indeed numbered with v1.4.537, but then the numbering of sections in the ToC is wrong (visible in your screenshot) and therefore the links do not work. Could it be related to the closed issue #1145?
And again, no issue with HTML. So it seems the issue is specific to GFM.

[cderv]

@dragonstyle this was introduced by #7533 (cc @cscheid ) to try fix #1145

by adding support for number-sections, it also added a duplicated numbering in TOC. So I believe this needs to be dealt for 1.4 as fixing a bug by introducing another does not seem right.

Either :

• We need to not add our TOC numbering to markdown output
  

  quarto-cli/src/resources/filters/crossref/sections.lua

  Lines 64 to 68 in 4fd435b

  	if _quarto.format.isHtmlOutput() or _quarto.format.isMarkdownWithHtmlOutput() then
  	el.content:insert(1, pandoc.Span(
  	stringToInlines(section),
  	pandoc.Attr("", { "header-section-number"})
  	))

• We need post processor like in HTML to remove what Pandoc is doing
  

  quarto-cli/src/format/html/format-html.ts

  Lines 715 to 723 in 4fd435b

  	// remove toc-section-number if we have provided our own section numbers
  	const headerSections = doc.querySelectorAll(".header-section-number");
  	for (let i = 0; i < headerSections.length; i++) {
  	const secNumber = headerSections[i] as Element;
  	const prevElement = secNumber.previousElementSibling;
  	if (prevElement && prevElement.classList.contains("toc-section-number")) {
  	secNumber.remove();
  	}
  	}

It seems to me the first is what we should do.

[cscheid]

We need to not add our TOC numbering to markdown output

We do need that, though, for #1145. Am I missing something?

[cderv]

See my example at #1145 (comment)

• We have the number on the TOC
• We don't have it on the sections.

You can try the pre-release before the change you made (1.4.489) on

---
title: "toc-gfm-numbered-sections"
format:
  gfm: 
    toc: true
    number-sections: true
---

## H1

### Sub-H1

## H2

I am still trying to figure out what in the history we miss. It seems we assumes that --number-sections is supported by Pandoc directly for Markdown format

quarto-cli/src/command/render/pandoc.ts

Lines 828 to 842 in 7769ea9

	// remove selected args and defaults if we are handling some things on behalf of pandoc
	// (e.g. handling section numbering). note that section numbering is handled by the
	// crossref filter so we only do this if the user hasn't disabled the crossref filter
	if (
	!isLatexOutput(options.format.pandoc) &&
	!isTypstOutput(options.format.pandoc) &&
	!isMarkdownOutput(options.format) && crossrefFilterActive(options)
	) {
	delete allDefaults[kNumberSections];
	delete allDefaults[kNumberOffset];
	const removeArgs = new Map<string, boolean>();
	removeArgs.set("--number-sections", false);
	removeArgs.set("--number-offset", true);
	pandocArgs = removePandocArgs(pandocArgs, removeArgs);
	}

It seems to be like that since a long time (8875ec6)

But looking at Pandoc's Manual and trying out bare pandoc, I don't think Pandoc does number the section in markdown output like GFM / commonmark. It will only number the TOC.

> pandoc --to gfm --number-sections --toc -s
# Header 1
# Header 2
^Z
- [<span class="toc-section-number">1</span> Header 1](#header-1)
- [<span class="toc-section-number">2</span> Header 2](#header-2)

# Header 1

# Header 2
> pandoc --to commonmark --number-sections --toc -s
# Header 1
# Header 2
^Z
- [<span class="toc-section-number">1</span> Header 1](#header-1)
- [<span class="toc-section-number">2</span> Header 2](#header-2)

# Header 1

# Header 2

In R Markdown, we do support number_sections: true for markdown output through a Lua filter.

So my understanding is that we need to handle number section with our Lua filter for Markdown output, but not the TOC numbering as it seems to be supported.

[cderv]

Also, I don't think section were numbered in 1.3, 1.2 or even 1.1. So this does not seem like a regression. Just something that was supposed to be supported but does not seem to be and support added in #7533 created the number duplication in TOC