Pyright consuming 100% CPU

[larsks]

Describe the bug

I am running pyright under neovim lsp. When pyright is running, I see:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 791059 lars      20   0 3072068   2.5g  37180 R 117.6   3.9  10:08.94 node

Examining the pyright process with strace, it appears to start by calling statx() on every single file in my home directory, which consumes an enormous amount of resources. This behavior is independent of the directory from which neovim was launched.

Even after it has finished scanning files it continues to consume 100% CPU; at this point strace shows lots of calls to futex() interspersed with calls to getpid() (about 35,000 times in 10 seconds).

To Reproduce

1. Configure neovim to use pyright:

   require'lspconfig'.pyright.setup{
       on_attach=on_attach,
   }
   

   Where on_attach just sets up a bunch of key mappings.

2. Open a Python file with neovim.

Expected behavior

I expect pyright to check the syntax of the files I have open in neovim and nothing else.

Additional information

This is similar to #128 but @erictraut asked for a new issue for related problems.

[erictraut]

I can think of two potential explanations for the behavior you're describing:

1. You have configured an include path that points to your home directory (which effectively tells pyright to include all files in your home directory as part of your project).
2. You have a symbolic link somewhere in your project directory that references your home directory. That would have a similar effect, as pylance will attempt to follow the symbolic link as it is enumerating the Python files that comprise your project.

I recommend enabling "verboseOutput": true in the pyrightconfig.json file and then looking at the output for clues.

Am I correct in assuming that this happens only for a particular project, or do you see the same behavior if you create a new project (i.e. a new directory with some Python files in it) and open it in your editor?

[larsks]

Am I correct in assuming that this happens only for a particular project, or do you see the same behavior if you create a new project (i.e. a new directory with some Python files in it) and open it in your editor?

Just to be clear, I'm working with Neovim so I'm not working in the context of a "project". The symptoms are independent of which files I open or in which directory I am operating -- I get the same behavior when opening Python files that belong to a Git repository and for standalone files that are not part of any project.

You have configured an include path that points to your home directory (which effectively tells pyright to include all files in your home directory as part of your project).

I have not configured an include path anywhere.

You have a symbolic link somewhere in your project directory that references your home directory. That would have a similar effect, as pylance will attempt to follow the symbolic link as it is enumerating the Python files that comprise your project.

This behavior is independent of the directory in which I am operating, and there are no symlinks in any of the directories I am working with.

I recommend enabling "verboseOutput": true in the pyrightconfig.json file and then looking at the output for clues.

I will give this a try.

[larsks]

Actually, it looks like the behavior may be triggered by the absence of a pyrightconfig.json file. If I create pyrightconfig.json in a directory with the content:

{}

Then opening files in this directory doesn't trigger the problem.

[erictraut]

Do you have a pyrightconfig.json file in your home directory? If you open a file (as opposed to a directory / project / workspace / or whatever term neovim uses) pyright will scan up the directory hierarchy to find a pyrightconfig.json file. If it finds one, it assumes the opened file is part of a project rooted at that place in the directory hierarchy.

The logs should tell you which pyrightconfig.json file it is using.

[larsks]

I do not have a pyrightconfig.json in my home directory:

$ find $HOME -name pyrightconfig.json
/home/lars/tmp/python/pyrightconfig.json

(And that one match is the file I created just a minute ago.)

And just to confirm, if I open the (empty) file ~/tmp/foo.py...

cd ~/tmp
strace -o trace -f -s 80 nvim foo.py

I see:

824183 statx(AT_FDCWD, "/home/lars/tmp/foo.py/pyrightconfig.json", AT_STATX_SYNC_AS_STAT, STATX_ALL, 0x7fff24b93490) = -1 ENOENT (No such file or directory)
824183 statx(AT_FDCWD, "/home/lars/tmp/pyrightconfig.json", AT_STATX_SYNC_AS_STAT, STATX_ALL, 0x7fff24b93490) = -1 ENOENT (No such file or directory)
824194 access("/home/lars/pyrightconfig.json", F_OK) = -1 ENOENT (No such file or directory)

[erictraut]

I've never used neovim and don't know anything about it. Is there no way to open a directory / project / workspace? That's the recommended way to use pyright in VS Code. The language server protocol has the notion of a "workspace" with a directory root, so I assume that neovim supports this concept. You may need to seek technical support from the maintainers of the neovim LSP plugin for pyright.

For diagnostic purposes, we could temporarily eliminate neovim from the equation by leveraging the command-line version of pyright and see if that repros the problem you're seeing. You can pass --verbose on the command line to ask pyright to emit verbose logs. That eliminates the need to create a pyrightconfig.json file to get verbose output.

[larsks]

Simply running pyright from the command line does not appear to reproduce the problem.

It looks like neovim may be starting pyright from $HOME, which may explain the behavior. There is a root_dir option that looks like it might impact this behavior.

It looks as if configuring pyright like this (from neovim/nvim-lspconfig#1851)...

require'lspconfig'.pyright.setup{
    on_attach=on_attach,
    root_dir = function()
      return vim.fn.getcwd()
    end,
}

...configures it to start in neovim's cwd, rather than in $HOME, leading to more reasonable behavior. Hopefully other folks will find this useful!

[erictraut]

Ah, that would explain it. You might want to report this issue to the maintainers of the neovim LSP plugin for pyright. Configuring pyright to treat the entire home directory as a workspace will inevitably lead to problems like this. If they can prevent this, that would be good.

[larsks]

neovim/nvim-lspconfig#2238