Markdown missing header mark highlights

[polirritmico]

Describe the bug

After #6232 markdown files don't have a header marker highlight. For example, at the # Header 1, Inspect shows @markup.heading.1.markdown for all the line including the # marker and the Header 1 content text.
Before it shows @markup.heading.1.marker.markdown for the # mark and @markup.heading.1.markdown for the content.

Screenshot:

To Reproduce

1. Open a markdown file
2. :TSEnable markdown

Expected behavior

Provide a highlight to differentiate the marker from the text content.

Output of :checkhealth nvim-treesitter

nvim-treesitter: require("nvim-treesitter.health").check()

Installation ~
- OK `tree-sitter` found 0.20.8 (parser generator, only needed for :TSInstallFromGrammar)
- OK `node` found v20.11.0 (only needed for :TSInstallFromGrammar)
- OK `git` executable found.
- OK `cc` executable found. Selected from { vim.NIL, "cc", "gcc", "clang", "cl", "zig" }
  Version: cc (Gentoo 13.2.1_p20240113-r1 p12) 13.2.1 20240113
- OK Neovim was compiled with tree-sitter runtime ABI version 14 (required >=13). Parsers must be compatible with runtime ABI.

OS Info:
{
  machine = "x86_64",
  release = "6.6.13-gentoo",
  sysname = "Linux",
  version = "#1 SMP PREEMPT_DYNAMIC Mon Jan 22 13:30:15 -03 2024"
} ~

Parser/Features         H L F I J
  - bash                ✓ ✓ ✓ . ✓
  - c                   ✓ ✓ ✓ ✓ ✓
  - comment             ✓ . . . .
  - css                 ✓ . ✓ ✓ ✓
  - diff                ✓ . . . .
  - gitcommit           ✓ . . . ✓
  - html                ✓ ✓ ✓ ✓ ✓
  - json                ✓ ✓ ✓ ✓ .
  - jsonc               ✓ ✓ ✓ ✓ ✓
  - latex               ✓ . ✓ . ✓
  - lua                 ✓ ✓ ✓ ✓ ✓
  - luadoc              ✓ . . . .
  - luap                ✓ . . . .
  - make                ✓ . ✓ . ✓
  - markdown            ✓ . ✓ ✓ ✓
  - markdown_inline     ✓ . . . ✓
  - python              ✓ ✓ ✓ ✓ ✓
  - query               ✓ ✓ ✓ ✓ ✓
  - regex               ✓ . . . .
  - sql                 ✓ . . ✓ ✓
  - toml                ✓ ✓ ✓ ✓ ✓
  - vim                 ✓ ✓ ✓ . ✓
  - vimdoc              ✓ . . . ✓
  - yaml                ✓ ✓ ✓ ✓ ✓

  Legend: H[ighlight], L[ocals], F[olds], I[ndents], In[j]ections
         +) multiple parsers found, only one will be used
         x) errors found in the query, try to run :TSUpdate {lang} ~

Output of nvim --version

nvim --version
NVIM v0.9.5
Build type: Release
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/x86_64-pc-linux-gnu-gcc  -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wvla -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -fno-common -Wno-unused-result -Wimplicit-fallthrough -fdiagnostics-color=always -fstack-protector-strong -DUNIT_TESTING -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -I/usr/include/luajit-2.1 -I/usr/include -I/usr/include -I/var/tmp/portage/app-editors/neovim-0.9.5/work/neovim-0.9.5_build/src/nvim/auto -I/var/tmp/portage/app-editors/neovim-0.9.5/work/neovim-0.9.5_build/include -I/var/tmp/portage/app-editors/neovim-0.9.5/work/neovim-0.9.5_build/cmake.config -I/var/tmp/portage/app-editors/neovim-0.9.5/work/neovim-0.9.5/src -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include

archivo "vimrc" del sistema: "/etc/vim/sysinit.vim"
predefinido para $VIM: "/usr/share/nvim"

Run :checkhealth for more info

Additional context

I'm doing the report from 0.9.5 but I've also tested in 0.10 nightly with the same results.

Edit: Improve the description

[clason]

Deliberate. The capture names and highlight groups were standardized, which meant that the markers are no longer given a separate capture. If you want them, revert to the latest tagged commit, or re-add them in you personal config.

[polirritmico]

Why the difference between headers marks and e.g. list markers?

[clason]

Because 1. list markers don't get a special highlight group and 2. you don't typically highlight list entries, so only list markers need a highlight.

[polirritmico]

OK. From my user point of view, this seems like a regression, but as you pointed out, I could re-add them in my personal config.

Thanks for the explanation.

[clason]

It is a regression, but necessary (and justifiable) for maintainability.

[camoz]

What is the long-term plan regarding this? Will some of the removed capture names (i.e. those that didn't get a replacement, like @text.title.*.marker.markdown) eventually be added upstream? Or is it expected that users solve this on their own, maybe through the emergence of new plugins that provide additional capture names and highlight groups for certain languages?

[clason]

You are welcome to add your own queries. There is no plan to change this, either here or in Neovim.

[polirritmico]

For those who want this feature, this should bring back those highlights. In the nvim/after/queries/markdown/highlights.scm file:

;; extends
(atx_heading (atx_h1_marker) @markup.heading.1.marker)
(atx_heading (atx_h2_marker) @markup.heading.2.marker)
(atx_heading (atx_h3_marker) @markup.heading.3.marker)
(atx_heading (atx_h4_marker) @markup.heading.4.marker)
(atx_heading (atx_h5_marker) @markup.heading.5.marker)
(atx_heading (atx_h6_marker) @markup.heading.6.marker)

(setext_heading (setext_h1_underline) @markup.heading.1.marker)
(setext_heading (setext_h2_underline) @markup.heading.2.marker)