Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Treesitter integration #32

Closed
wookayin opened this issue Feb 2, 2022 · 4 comments
Closed

Treesitter integration #32

wookayin opened this issue Feb 2, 2022 · 4 comments

Comments

@wookayin
Copy link
Owner

wookayin commented Feb 2, 2022 •
edited
Loading

Continued from #27

Basic support (syntax, folding)

  • ✅ https://github.com/nvim-treesitter/nvim-treesitter
  • ✅ Debugging: https://github.com/nvim-treesitter/playground
  • Note: Treesitter's syntax highlight overrides existing python syntax, which is annoying It should also replace semshi (since most use cases are covered). It also breaks some other plugins that rely on highlight groups (e.g. vim-autoimport) because synID cannot retrieve dynamic highlights. When TS is enabled, old vim regex highlights should be avoided (they usually don't work together nicely).

Some cool plugins powered by treesitter

See https://github.com/nvim-treesitter/nvim-treesitter/wiki/Extra-modules-and-plugins for a comprehensive list
wookayin added a commit that referenced this issue Feb 5, 2022
@wookayin
vim: Add treesitter (#32)
83f3b9b 
This is a minimal setup of nvim-treesitter with the playground plugin
in addition. Treesitter-based highlights are disabled, so it should
have no dramatic changes from users' perspective.
wookayin added a commit that referenced this issue Feb 5, 2022
@wookayin
vim: Change the alias of treesitter-playground plugin (#32)
8c9326c 
- treesitter-playground -> nvim-treesitter-playground
- To make the name consistent with other treesitter-based plugins.
wookayin added a commit that referenced this issue Feb 8, 2022
@wookayin
vim: Do not lazy-load treesitter
6b1b8f7 
Some lua-based plugins won't work properly when lazy-loaded by
vim-plug. A good workaround is to not lazy load treesitter,
at the expense of a little bit increased startup time.

Ref: #32
wookayin added a commit that referenced this issue Feb 8, 2022
@wookayin
vim: Update treesitter-playground and auto-install config
f9b36e3 
Ref: #32
wookayin added a commit that referenced this issue Feb 9, 2022
@wookayin
vim: Make sure treesitter updates parse tree when text changes (#32)
ec2f252 
In the current implementation of treesitter, the parse tree for the
current buffer would not be automatically updated/refreshed when
text changes, unless treesitter-highlight module is enabled.
However, I don't want to use treesitter's highlight/syntax feature yet
due to some issues and conflicts.

To make the parse tree up-to-date as text changes, parser:parse()
needs to be explicitly called in the absence of treesitter-highlight.

Ref: nvim-treesitter/nvim-treesitter#2492
@wookayin
Copy link
Owner Author

wookayin commented Feb 9, 2022

To make (some) treesitter modules work properly with up-to-date parse tree as text change, either playground-highlight module needs to be enabled or some workaround like ec2f252 is needed, when one doesn't want to use playground's highlight/syntax feature yet.

wookayin added a commit that referenced this issue Apr 16, 2022
@wookayin
vim: Improve treesitter queries for python folding (#32)
69d5f7a 
We replace the existing treesitter query for python folding with
a custom query file.

This allows to fix the undesired behavior of folding the body of
function/class definitions (or other block statements).
We would want to be able to fold the whole statement.

Ref: nvim-treesitter/nvim-treesitter#2804
@wookayin
Copy link
Owner Author

Improved python folding -- the whole function/class is now foldable, rather than its body only (which should also be fixed in the upstream soon).

wookayin added a commit that referenced this issue Apr 19, 2022
@wookayin
vim: Make python import statements foldable
a0b650b 
Write a custom, extended treesitter query to capture the region of
python import statements.

Ref: #32
@wookayin
Copy link
Owner Author

wookayin commented Jan 6, 2023 •
edited
Loading

Some issues with treesitter highlights (for lua): embedded vimscripts vim.cmd [[ ... ]] are highlighted as @string:

image

-> Workaround discussed in https://www.reddit.com/r/neovim/comments/1059xht/comment/j3dw7ty/

image

See 5a4b971

wookayin added a commit that referenced this issue Jan 8, 2023
@wookayin
vim(ts): Use treesitter highlight for lua files
5a4b971 
Neovim's lua syntax written in vimscript is broken since neovim 0.8
(see neovim issue 20456); as a workaround, we use treesitter-based
highlight for lua files at the moment. To deal with vimscript injections
properly, we would also need a custom user queries to fix incorrect
highlights on "injected" @string.

References:

- https://www.reddit.com/r/neovim/comments/1059xht/how_to_correctly_highlight_injected_vimscript/
- #32 (Treesitter integration)
wookayin added a commit that referenced this issue May 10, 2023
@wookayin
vim(treesitter): Use treesitter highlight for vimscript and lua
baa7096 
The lua syntax file in neovim 0.8.0+ is broken. Now that we have
neovim >= 0.8, we can start using treesitter syntax highlighting for
at least vimscript and lua which seem to work OK so far.

See #32 for more context and history.
wookayin added a commit that referenced this issue Sep 19, 2023
@wookayin
vim(treesitter): Refactor treesitter config as a module
bc5ea75 
- Make config.treesitter as a lua module
- Miscellaneous style improvements
- Automatic TS parsing upon text change is only needed for nvim <= 0.9

Ref: #32
wookayin added a commit that referenced this issue Sep 19, 2023
@wookayin
vim(treesitter): Remove treesitter parsing trigger workaround
0995362 
Remove the workaround introduced in ec2f252 (#32) when neovim was ~0.7.
This was needed because treesitter's incremental parsing did not happen
when the highlight module was not enabled. This might have been fixed
by neovim core (for 0.8+) or by the nvim-treesitter plugin (not sure),
but incremental parsing now works OK even when treesitter highlight
is not enabled, so we probably no longer need this workaround.
wookayin added a commit that referenced this issue Sep 19, 2023
@wookayin
vim(treesitter): Add setup_highlight and ensure_parsers_installed
be7102e 
- `M.setup_highlight`: similarily to `vim.treesitter.start()`, it can be
  used to manually enable treesitter highlighter. Because the treesitter
  API may throw an error when parsers are outdated or ill-configured,
  we add a safeguard `M.has_parser(...)` to check whether a TS parser
  instance can be actually obtained (it's something more than done by
  `ts_parsers.has_parser`). Ftplugins should use this function instead
  of callling `vim.treesitter.start()`

- `M.ensure_parsers_installed`: Request to install treesitter parsers
  asynchronously if any of the parsers haven't been installed yet.
  It can be used to install parsers dynamically upon loading buffers
  of a specific filetype for the first time, rather than listing
  a number of parsers to install in `ts_configs.ensure_installed`.

See #32.
wookayin added a commit that referenced this issue Sep 19, 2023
@wookayin
vim(treesitter): Exclude neovim's built-in TS parsers for has_parser
529c4ae 
Neovim 0.8.0+ ships with built-in TS parser modules (*.so) in the lib
path, e.g., `/usr/local/lib/nvim/` if `$VIM` is /usr/local/share/nvim/`,
which conflicts with TS plugins usually installed *lazily* through
the nvim-treesitter plugin. Normally such a conflict would have no big
problem if treesitter parsers have already been installed, and is
loadable as the first available parser from `&runtimepath`. However,
this could cause some errors if nvim-treesitter parsers were yet to be
installed, as neovim's built-in TS parsers would be loaded and used
instead of downloaded nvim-treesitter parsers, but then the actual
runtime queries sourced by the plugin will be incompatible with the
built-in TS parsers.

One solution is to exclude neovim core's built-in TS parsers from the
candidates for `has_parser(lang)`. This function is called in my config
to determine whether treesitter parsers are installed and ready to work,
in order to activate treesitter highlights manually, for instance.
`nvim-treesitter.parsers.has_parser(...)` may also take a similar
approach, but currently it does not exclude neovim's built-in parsers.

See also: nvim-treesitter/nvim-treesitter#3970
See also: #32
wookayin added a commit that referenced this issue Sep 19, 2023
@wookayin
vim(treesitter): Enable TS highlight later when parsers arrive late
872a99f 
Problem: Because we enable treesitter highlight by manually calling
vim.treesitter.start() only if the parsers are available (see 3c34b0e),
the buffer will have no highlights until the requested parsers are
installed lazily and later asynchronously.

Solution: Register a dummy TS module to nvim-treesitter so that
we can manually start treesitter highlighting again, when the
nvim-treesitter module "re-attachs" as soon as the requested treesitter
parsers get succesfully installed and become available.

Related: #32
wookayin added a commit that referenced this issue Sep 20, 2023
@wookayin
vim(treesitter): Fix perf regression on has_parser, slow startup
a937232 
M.has_parser should not perform any TS parsing but respond quite
instantly, otherwise it can block upon loading (large) buffers.

Actually, it was trying to load python parsers due to a has_parser()
call in load_custom_query().

Ref: #32
@wookayin
Copy link
Owner Author

wookayin commented Dec 9, 2023 •
edited
Loading

  • Most of core filetypes (lua, cpp, vim, rust, tex, etc.) now use treesitter highlights (see be7102e). See individual filetype plugins.
  • Automatic parser installation and error recovery utils are added.
  • Some treesitter utilities: 91b77f0.

Closing as done. Configurations for advanced plugins will be added in the next steps.

@wookayin wookayin closed this as completed Dec 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wookayin