Highlight definition can be slow and blocks cursor movement

[theHamsta]

Describe the bug
highlight_definitions can be really slow if it does not find the definition in current scope but in one of the parent levels.
This can be annoying since it blocks cursor movement while it is running.

To Reproduce
Open a big Python or C++ file with a lot of variable definitions (like this one https://i10git.cs.fau.de/pycodegen/pystencils/-/raw/master/pystencils/field.py)

Try to move over the individual letters of @staticmethod. It will not find a definition but quite a few usages. Cursor movement is blocked during the search of the definitions.

Expected behavior
Cursor movement is smooth. Search for definitions is non-blocking

** Output of :checkhealth nvim_treesitter **

```

health#nvim_treesitter#check

Installation

• OK: git executable found.
• OK: cc executable found.

html parser healthcheck

• OK: html parser found.
• WARNING: No locals.scm query found for html
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

typescript parser healthcheck

• OK: typescript parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

clojure parser healthcheck

• OK: clojure parser found.
• WARNING: No locals.scm query found for clojure
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for clojure
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

lisp parser healthcheck

• OK: lisp parser found.
• WARNING: No locals.scm query found for lisp
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

regex parser healthcheck

• OK: regex parser found.
• WARNING: No locals.scm query found for regex
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

c parser healthcheck

• OK: c parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

nix parser healthcheck

• OK: nix parser found.
• WARNING: No locals.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

swift parser healthcheck

• OK: swift parser found.
• WARNING: No locals.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

java parser healthcheck

• OK: java parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

python parser healthcheck

• OK: python parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

yaml parser healthcheck

• OK: yaml parser found.
• WARNING: No locals.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

elm parser healthcheck

• OK: elm parser found.
• WARNING: No locals.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

vue parser healthcheck

• OK: vue parser found.
• WARNING: No locals.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

cpp parser healthcheck

• OK: cpp parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

toml parser healthcheck

• OK: toml parser found.
• WARNING: No locals.scm query found for toml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for toml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

lua parser healthcheck

• OK: lua parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

ruby parser healthcheck

• OK: ruby parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

ocaml parser healthcheck

• OK: ocaml parser found.
• WARNING: No locals.scm query found for ocaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for ocaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

go parser healthcheck

• OK: go parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

scala parser healthcheck

• OK: scala parser found.
• WARNING: No locals.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

haskell parser healthcheck

• OK: haskell parser found.
• WARNING: No locals.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

rust parser healthcheck

• OK: rust parser found.
• WARNING: No locals.scm query found for rust
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

json parser healthcheck

• OK: json parser found.
• WARNING: No locals.scm query found for json
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

markdown parser healthcheck

• OK: markdown parser found.
• WARNING: No locals.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

javascript parser healthcheck

• OK: javascript parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

css parser healthcheck

• OK: css parser found.
• WARNING: No locals.scm query found for css
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

julia parser healthcheck

• OK: julia parser found.
• WARNING: No locals.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

php parser healthcheck

• OK: php parser found.
• WARNING: No locals.scm query found for php
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for php
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

c_sharp parser healthcheck

• OK: c_sharp parser found.
• WARNING: No locals.scm query found for c_sharp
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No highlights.scm query found for c_sharp
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

bash parser healthcheck

• OK: bash parser found.
• WARNING: No locals.scm query found for bash
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: highlights.scm found.

tsx parser healthcheck

• OK: tsx parser found.
• OK: locals.scm found.
• OK: highlights.scm found.

Output of nvim --version

➜ nvim --version
NVIM v0.5.0-bd5f0e969
Build type: Release
LuaJIT 2.0.5
Compilation: /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/linux/super/gcc-10 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNDEBUG -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/tmp/neovim-20200704-130349-1xlbcrp/build/config -I/tmp/neovim-20200704-130349-1xlbcrp/src -I/home/linuxbrew/.linuxbrew/include -I/tmp/neovim-20200704-130349-1xlbcrp/deps-build/include -I/usr/include -I/tmp/neovim-20200704-130349-1xlbcrp/build/src/nvim/auto -I/tmp/neovim-20200704-130349-1xlbcrp/build/include
Compiled by stephan@stephan-Z87-DS3H

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "
/home/linuxbrew/.linuxbrew/Cellar/neovim/HEAD-bd5f0e9_1/share/nvim"

Run :checkhealth for more info

Additional context
Add any other context about the problem here.

[vigoux]

Oh we can move that as an async method, to avoid blocking, maybe...
It will need to be cancellable though

[steelsojka]

@vigoux What's the nvim/lua api for making this async? I can take a look at this.

[vigoux]

@steelsojka, I never used it personnally, but I think that the help page is located at :h lua-loop.

[theHamsta]

I think we just wait for lua support for jobs. There is a PR for it in Neovim. I think it was called vim_wrap or something like that. Can give you hints tomorrow.

You would start the async job with a counter and only perform the highlighting when its finished when the counter was not incremented in the mean time. This is easier to implement that directly using vim.loop.

[steelsojka]

Most of the performance issue is coming from checking if the node is a reference when recursing over the tree.

  M.recurse_tree(scope_node, function(iter_node, _, next)
    if vim.tbl_contains(references, iter_node) and M.get_node_text(iter_node)[1] == node_text then
      table.insert(usages, iter_node)
    end
    next()
  end)

I'll find a better way to check if it's a reference

[steelsojka]

I'll also make it async as well 👍

[steelsojka]

#154 This fixes a lot of the slowness when I was testing. I will explore making the api async as well, but this is probably a good solution for now.

[vigoux]

Is the issue still happening with latest master? If not, we will close this.