[Enhancement] Add support for semantic token highlighting.

[csy19960309]

Enhancement

It would be nice if onedarkpro.nvim can offer syntax highlights with the help of LSP.
Semantic token highlighting has been merged into the master branch of neovim neovim/neovim@9f03555.

[olimorris]

Completely agree. This is was raised by @mmirus in #82 (comment) and something that we will definitely incorporate.

Reading through the commits, it looks like some new highlight groups are by default, linked to vim syntax groups.

--LSP semantic tokens
"default link @class Structure",
"default link @struct Structure",
"default link @enum Type",
"default link @enumMember Constant",
"default link @event Identifier",
"default link @interface Identifier",
"default link @modifier Identifier",
"default link @regexp SpecialChar",
"default link @typeParameter Type",
"default link @decorator Identifier",

So should be very easy to incorporate. Also, that will give us some absolutely incredible highlighting. I'm running nvim 0.8 at present so if someone wants to be beat me to it, I'll be happy to accept a PR.

[olimorris]

I've just had a play with Neovim 0.9 (no config breaking changes so far btw). I see there's a PR for Treesitter Playground which will make it easier to identify which tokens are being used for what. Propose we wait for that before adding support.

[csy19960309]

I've just had a play with Neovim 0.9 (no config breaking changes so far btw). I see there's a PR for Treesitter Playground which will make it easier to identify which tokens are being used for what. Propose we wait for that before adding support.

All right. but having to install Treesitter to make use of semantic tokens seems to be a little bit unnecesarry. Hope some kind guy can give a PR so that we can enjoy that feature directly with LSP :)

[olimorris]

All right. but having to install Treesitter to make use of semantic tokens seems to be a little bit unnecesarry. Hope some kind guy can give a PR so that we can enjoy that feature directly with LSP :)

No you don't it's built into the core of Neovim and as long as your LSP server supports semantic tokens, you'll get the highlighting.

Treesitter Playground is a separate plugin which has functionality allowing you to identify which highlight groups Treesitter is applying to your code. There's a PR to extend it so it can do the same for LSP semantic tokens. I want to use Playground to allow me to see where Neovim applies the @class semantic token, for example, and therefore what color it should be in the theme. I could probably guess but I suspect the PR will be merged in the next couple of days.

[csy19960309]

All right. but having to install Treesitter to make use of semantic tokens seems to be a little bit unnecesarry. Hope some kind guy can give a PR so that we can enjoy that feature directly with LSP :)

No you don't it's built into the core of Neovim and as long as your LSP server supports semantic tokens, you'll get the highlighting.

Treesitter Playground is a separate plugin which has functionality allowing you to identify which highlight groups Treesitter is applying to your code. There's a PR to extend it so it can do the same for LSP semantic tokens. I want to use Playground to allow me to see where Neovim applies the @class semantic token, for example, and therefore what color it should be in the theme. I could probably guess but I suspect the PR will be merged in the next couple of days.

Well I thought the playground was what I had to install to enable the highlighting, sorry for the misunderstanding. Now we just wait for that PR the be merged :)

[olimorris]

Quick update on this:

• Semantic tokens are now merged in Neovim 0.9 as per #21100
• We now have the :Inspect command in Neovim 0.9 which shows the origin of highlight groups, be them Treesitter or semantic tokens, as per: #21393
• The issue of modifiers (brilliantly described in #21576) is still to follow

Implementation of the tokens is a simple case of adding in the new highlight groups as indicated above. I'll give this some time as want to do some reconciliation back to VS Code but will likely implement in the next month.

EDIT: Link to semantic highlights in Neovim core

[olimorris]

In 0e64181 I've started to define semantic tokens. I've been restrained in the tokens I've defined so far (matching to sumneko_lua in VS Code) and the results are impressive:

Before

After

Visual Studio Code

I'm planning on overhauling the way styles are applied in the next week or two.

[csy19960309]

In 0e64181 I've started to define semantic tokens. I've been restrained in the tokens I've defined so far (matching to sumneko_lua in VS Code) and the results are impressive:

Before

After

Visual Studio Code

I'm planning on overhauling the way styles are applied in the next week or two.

I updated to the latest commit and it works like a charm. Thanks for the work!

[olimorris]

I'll close this for now but expect to add more highlights over time