feat!: rewrite TOhtml in lua

[altermo]

The reasoning behind the rewrite

1. Some neovim-specific decorations (mainly extmarks and treesitter) don't get handled by TOhtml.vim.
2. TOhtml.vim is written in Vim script, and to not make it slow, they needed to write some hard-to-understand code, and lua is so much faster, that there is not much need to use magic to write fast code.
3. TOhtml.vim is not updated that often (typically once every few years) so there's no need to worry about rewriting every new feature of TOhtml.vim in lua.

What this rewrite is not
As @clason described, this rewrite needs not to be a full rewrite of TOhtml.vim (with all the features), so it is not.

What this rewrite is
This rewrite has the purpose of creating a plugin which can convert a vim window with all it's decorations to html.

The visual things that still don't work

• LSP semantic tokens that are not on the screen
• Highlighted folds
• More complex internal statuscol (number+relativenumer+{foldcolumn or signcolumn})
• Right aligned extmarks
• Extmarks' hl_mode
• Spell
• Priority for specific decorations
• Less used highlight attributes (font, special color, reverse, underdouble, blend, ...)

Related:

• add :Hardcopy command #21629

[clason]

Question to the Neovim maintainer: Do we need to support everything that TOhtml.vim does?

Ideally, yes. The assumption should be that the output is exactly what's seen on the screen. But it's ok not to support everything in a first MVP that can be merged, if the design is such that it can be added in followup PRs.

This also implies that we don't need to support every feature and option from the legacy :TOhtml (which users can still install as a plugin if they pine for the good old days). Focus on the primary use case (hardcopy/offline viewing), and only implement what is needed for that. (This is an opportunity for simplification and better maintainability, importantly.)

Specifically:

1. linking URLs would be good but not needed for an MVP;
2. no progress bar (Lua is hopefully fast enough that you don't need one);
3. no interactivity/lots of settings (just set the right Vim options before :TOhtml, which gives you a preview for free);
4. no legacy web engines, no legacy encodings (Neovim is utf-8 only).

[clason]

I think it's OK to keep the syntax file (and even preferable; makes syncing with Vim runtime slightly easier).

[clason]

Nicely done! Thank you for tackling this!

[clason]

(Also needs a rebase.)

[altermo]

I accidentally messed up the commit, how to fix it? (did to fast, need to go somewhere, don't have time to fix)

[clason]

Yeah, I'm afraid that you messed it up properly. Disentangling will take some time and effort. You need to git restore to "open up" the commit and carefully stage all your changes before committing again.

[clason]

(As far as I can see, only gen_vimdoc.py returning from the dead is spurious, so you might get away with git rming that and squashing again. But you should carefully look at the diff to make sure nothing got lost.)

[altermo]

Okay, fixed the commit.

[clason]

I just noticed a minor issue: the newly opened buffer with the HTML inherits filetype=help instead of setting it to html.