[BUG] Syntax highlighting is slow and inefficient.

[amano-kenji]

Version of Kakoune

v2021.11.08

Reproducer

• Try to open any file with 20,000 or more lines.
• Try multi selection editing in the file.

Outcome

For files that have 20,000 lines or more, syntax highlighting is slow.

The file size is still less than 1 megabytes with a bit more than 20,000 lines.

Multi selection editing is also slow due to syntax highlighting. Perhaps, syntax highlighting is updated as I type each letter in editing mode.

neovim doesn't seem to apply syntax highlighting to the whole file. It seems to apply syntax highlighting to just visible parts. But, neovim's syntax highlighting is buggy because it applies syntax highlighting to visible parts only.

3D engines are efficient because they render only what they have to. Recent 3D engines even tweak level of detail dynamically to make rendering more efficient.

Expectations

Syntax highlighting should be efficient.

Additional information

No response

[mawww]

Would you have an example source file to reproduce this on ? Kakoune does only apply highlighting to the currently displayed part of the buffer and tries to aggressively cache data to avoid reparsing. The only highlighting logic that uses the full buffer is region handling but this should only parse the whole buffer once, then only apply to modified lines between highlighting updates.

[amano-kenji]

example.journal.txt

This is an example ledger journal file.

• Try to select Yeti with %sYeti[Enter] and edit it.

With ledger filetype, doing this takes a long time even with no hook(\). With plain filetype, doing this is very fast.

[mawww]

Took a look, this seems to be an issue in ledger.kak that defines highly costly highlighting (it seems to define regions that match every single character of the buffer)

[asb]

I'm seeing the same issue with a very large Markdown file (I keep all my notes in one big file). The ~8.5MiB file takes > 10s to display in Kakoune, though displays instantly if I rename it to .txt.

[mawww]

@asb would you have an example file you could provide for the markdown case ?

[asb]

It looks like there's nothing in particular special about my notes file. The following script generates a ~5MiB file that takes >5 seconds to open with Kakoune.

#!/bin/sh

for i in $(seq 10000); do
  cat <<EOF >> big_markdown.md
# Title
## Subtitle
Some notes about things.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore
eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt
in culpa qui officia deserunt mollit anim id est laborum

* A list
  * With a sub-item
* Item 2
* Item 3
* Item 4
* Item 5
  * Item 5.1

EOF
done