feat(concealer)!: rewrite for performance and stability

[champignoom]

I'm trying to solve the concealer problems (#821, #831, #833, etc.) systematically, but the current implementation is not very easy to deal with. Not few lines have large (up to 18) indentation levels.

neorg/lua/neorg/modules/core/concealer/module.lua

Line 386 in a295adf

- range.column_start,

This is a tracker PR of the refactoring of concealer and probably of the fixes as well.

Bug fixes

• fixes concealer broken when promoting first element #821
• fixes incomplete concealing #831
• fixes undo leaves concealer artifact #833
• fixes anchor definition not concealed when conceallevel=2 #838 (reopening wanted)
• fixes concealer ignoring insert mode movement #839
• fixes concealer support for ordered list level>6 #844
• fixes concealer missing boundary check for ordered list #846
• fixes incorrect conealing of spoiler #848
• should fix Copy and paste does not recognize devicon upon escape #892

[champignoom]

Features

It's so far instantly responsive, consistent, and mostly backward compatible. With a more precise tracking of changes with phantom extmarks on each line, only changed parts that falls within the dozens of currently visible lines are rerendered, so there's no more performance tweaks.

Breaking changes

• There's no more "enabled" in configs as it's rather redundant and makes the wiki noisy. To disable part of a config, change e.g. pending = { ... } to pending = false and { ... }
• There's no more list_1..6 or heading_1..6. To specify the icons, just write e.g. heading = { icons = { "!", "@", "#", "$" } }, where "!" is the icon for level 1, "@" for level 2, etc. The last one is used for remaining levels. Unlimited (in particular >6) number of levels can be specified. Same for ordered list, unordered list, and quote.
• The icons of ordered list are now specified with one of 1,⒈,⑴,①,a,A,⒜,Ⓐ,ⓐ, instead of something like parenthesis(latin).
• There's no more "performance" section in config, as performance tweaks are dropped altogether.

TODO

with decreasing urgency:

• fe601e8 BufRead is not triggered when editing new files. Another event is need. I'm not sure why/whether this hasn't been a problem.
• 881996f Adapting preset icons.
• More testing.
• Comments and documentation.
• d1e2f6a Fix the error during :h, related: Autocommand CursorMoved is called in confusing/misleading ways neovim/neovim#11878
• Error handling in configs.
• 3f1197a Rendering upon scrolling (ctrl-E/Y)
• A new command for rerendering, ideally bound to ctrl-L.
• Multi-column numbering for ordered lists using anticonceal.
• Roman numbering for ordered lists.
• Concealed quotation block doesn't render | on new lines #739

[max397574]

how does the performance compare to the old implementation?

[vhyrro]

Oh superb I'll get to reviewing this once i get home!

@max397574 champignoom says it's instantly responsive now, will test.

[max397574]

just tested it
it feels much faster than previous implementation
also many of the redrawing issues (after deleting, pasting, yanking, moving, undoing etc) no longer occur
only thing
it looks like the icon preset no longer works for me

[champignoom]

it looks like the icon preset no longer works for me

Fixed.

[vhyrro]

Fantastic work! The performance can really be felt - no stutters and no visual artifacts that I could find so far.

A few little things that I found while testing:

• Empty lines in code blocks are not highlighted
• Code block highlight disappears when moving over them in insert mode. This could be considered intended, but I'd argue that the old behaviour of retaining the highlight is the least jarring :p
• Regular {links} are no longer concealed. I assume this is because you conceal the heading parts of {* heading}, and that causes issues when the content is concealed? EDIT: Apparently this is the case in main too? I either have dementia or I vividly recall it working normally before.

This plus the other todos and this is looking really good!

[max397574]

I don't like the idea of having a default <c-l> mapping
this is such a common key to be mapped

[champignoom]

Fantastic work! The performance can really be felt - no stutters and no visual artifacts that I could find so far.

A few little things that I found while testing:

• Empty lines in code blocks are not highlighted
• Code block highlight disappears when moving over them in insert mode. This could be considered intended, but I'd argue that the old behaviour of retaining the highlight is the least jarring :p
• Regular {links} are no longer concealed. I assume this is because you conceal the heading parts of {* heading}, and that causes issues when the content is concealed? EDIT: Apparently this is the case in main too? I either have dementia or I vividly recall it working normally before.

This plus the other todos and this is looking really good!

The first two problems are fixed.

For the third one, could you give a concrete example of such link and the expected concealing behavior?

[aldanor]

I don't like the idea of having a default mapping
this is such a common key to be mapped

Indeed. IIRC, <c-l> is mapped by default by most nvim 'distributions' (e.g. in lazyvim, it's bound to "switch to the window to the right").

[max397574]

I haven't looked into how this works (and I won't)
but would it be possible to keep in mind that we got this now neovim/neovim#20130
we probably can't use it yet since we can't expect people to be on latest nightly but perhaps make it easy to be able to use it later (@vhyrro )

[champignoom]

This should be useable now.

Except that the configuration customization is still buggy and underdocumented. Something from the configuration system itself must first be fixed in another PR (e.g. #863, which is not a concealer problem)

[vhyrro]

Hope you don't mind, I just pushed an extra fix that does two things:

• Removes the necessity of placing two extmarks for code blocks
• Fixes a fun edge case bug with partially empty lines in code blocks, causing the concealer to just be confused.

I tested it pretty extensively and to my knowledge it doesn't introduce any bugs 😅.

After this I think I'll dive in to fix luacheck screaming and after that I'm fine with merging this module as it is :), we can introduce extra documentation and fixes as we go.

[vhyrro]

It's time to merge! @champignoom just wanted to say this is really impressive work, the concealer is something I myself wouldn't even want to touch with a metre long stick lol. Thank you :)