Intellisense unusable today

[zbs]

All the intellisense features (auto-complete, ctrl+click on symbols, etc) are not working today (they only show "Loading..."). In the Python Lanugage Server output, I see this:

[Info  - 1:31:28 PM] Pylance language server 2021.3.2 (pyright 85309906) starting
[Info  - 1:31:28 PM] Server root directory: /root_dir/path/to/dir
[Info  - 1:31:28 PM] No configuration file found.
[Info  - 1:31:28 PM] Setting pythonPath for service "workspace_root": "/home/zbs/path/to/python"
Search paths found for configured python interpreter:
    path1
    path2
    ...
[Error - 1:31:28 PM] stubPath /path/to/git/repo/typings is not a valid directory.
[Warn  - 1:31:28 PM] Python version 2.7 from interpreter is unsupported
[Info  - 1:31:28 PM] Assuming Python platform Linux
[Info  - 1:31:28 PM] Searching for source files
[Info  - 1:31:53 PM] Auto-excluding /path/to/git/repo/outer_dir/symlink_to_list_of_miniconda_installations/miniconda_installation_1

and many many more Auto-excluding lines.

I've tried:

• disabling search.followSymlinks
• removing outer_dir from the python.analysis.extraPaths
• adding outer_dir to files.exclude, files.watchExclude, search.exclude
  but none of them have worked.

Environment data

• Language Server version: 2021.3.2
• VS Cde: 1.54.3
• OS and version: Windows 10 Enterprise 20H2, using Remote SSH onto an RHEL 7 Host
• Python version (& distribution if applicable, e.g. Anaconda): Python 2.7 on Miniconda 3

[jakebailey]

It sounds a bit like you have a recursive symlink and we're getting stuck on that. The most recent release fixed a bug where our symlink support was doing the wrong thing, so it's possible we are recursing in some code path and never ending. Can you check to see if the paths printed in your logs have a structure like this?

[zbs]

We have possible symlink loops in under this subdirectory tree:

/path/to/git/repo/outer_dir/symlink_to_list_of_miniconda_installations

but no looping is indicated in the Auto-Excluding paths (e.g. I don't see the same path components in the filepaths).

Confirmed that the issue goes away when reverting to 2021.3.1.

[jakebailey]

Thanks.

[jakebailey]

I've submitted a PR which will likely fix this for you. microsoft/pyright#1653

[jakebailey]

This issue has been fixed in version 2021.3.3, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202133-24-march-2021

[zbs]

This issue seems to be happening again in 2021.3.4. The same console output in the original issue still applies. Switching back to 2021.3.1 fixes it again.

[jakebailey]

Okay. I'm a bit out of ideas here. The only think I can think to do would be to have you trace the file access to figure out where we're spinning (if the symlinks are the problem), as in: #1078 (comment)

[zbs]

When I strace the process, I get this:

lstat("/physical", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
lstat("/physical/gpfs", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
lstat("/physical/gpfs/us_west-devgroupusers", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                                                                      
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02", {st_mode=S_IFDIR|S_ISVTX|0775, st_size=262144, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                                          
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup", {st_mode=S_IFDIR|S_ISGID|0771, st_size=8192, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                              
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home", {st_mode=S_IFDIR|S_ISGID|0751, st_size=32768, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                        
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup", {st_mode=S_IFDIR|0750, st_size=8192, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                                             
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith", {st_mode=S_IFDIR|S_ISGID|0755, st_size=8192, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                        
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                    
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git", {st_mode=S_IFDIR|S_ISGID|0750, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                                
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name", {st_mode=S_IFDIR|S_ISGID|0755, st_size=8192, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support", {st_mode=S_IFLNK|0777, st_size=30, ...}) = 0                                                                                                                                                                                                                                                                                                                                 
stat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                        
readlink("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support", "../../../../../minimal/minimal", 4096) = 30                                                                                                                                                                                                                                                                                                                              
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release/codebase", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                               
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release/codebase/utils_package", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                       
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release/codebase/utils_package/utils_package", {st_mode=S_IFLNK|0777, st_size=30, ...}) = 0                                                                                                                                                                                                                                                                                                                                         
stat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release/codebase/utils_package/utils_package", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                
readlink("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/minimal/minimal/release/codebase/utils_package/utils_package", "../../../../../utils_package/utils_package", 4096) = 30                                                                                                                                                                                                                                                                                                                                      
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                           
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package/scheduler", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                                   
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package/scheduler/web", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                               
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package/scheduler/web/client", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                        
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package/scheduler/web/client/src", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0                                                                                                                                                                                                                                                                                                                                    
lstat("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/utils_package/utils_package/scheduler/web/client/src/main", {st_mode=S_IFDIR|S_ISGID|0755, st_size=4096, ...}) = 0
access("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support/release/codebase/utils_package/utils_package/scheduler/web/client/src/main/bin/activate", F_OK) = -1 ENOENT (No such file or directory)
access("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support/release/codebase/utils_package/utils_package/scheduler/web/client/src/main/Scripts/activate", F_OK) = -1 ENOENT (No such file or directory)
access("/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support/release/codebase/utils_package/utils_package/scheduler/web/client/src/main/pyvenv.cfg", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support/release/codebase/utils_package/utils_package/scheduler/web/client/src/main", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 24
getdents(24, /* 3 entries */, 32768)    = 80
getdents(24, /* 0 entries */, 32768)    = 0
close(24)                               = 0

It's worth noting that in this filepath:

/physical/gpfs/us_west-devgroupusers/us_west_devgroup_users02/data_files/devgroup/home/devgroup/zsilversmith/dev/git/repo_name/server_configuration/server_configuration/classes/db/home/devgroup.minimal.support/release/codebase/utils_package/utils_package/scheduler/

devgroup.minimal.support represents a symlink to the release version of the repo_name codebase. Did 2021.3.1 not resolve symlinks? If that's the case, then I suspect the problem in 3.4 is not a symlink loop, but symlinks resolving to really large directory subtrees that gum up the whole indexing process. A quickish fix would be to either disable symlink resolution in pylance, or to allow users to exclude directories from indexing. I think I prefer the latter because it gives users more control, and I think it's probably a pretty common use case that there are subtrees that don't need indexing.

[jakebailey]

Yes, 2021.3.2 was the first release that started adding symlink support.

It's not as simple as adding a toggle for this; lots of filesystem actions automatically follow links. Previously, some stuff worked, some stuff didn't.

Note that you can use a pyrightconfig.json to exclude specific directories, if they must be within your workspace root: https://github.com/Microsoft/pyright/blob/master/docs/configuration.md

[zbs]

okay, let me give that a shot. thanks

[zbs]

the pyright config doesn't seem to be preventing the bad paths from being searched:

[Info  - 2:23:25 PM] Pylance language server 2021.4.0 (pyright 2918da1d) starting
[Info  - 2:23:25 PM] Server root directory: /home/zsilversmith/.vscode-server/extensions/ms-python.vscode-pylance-2021.4.0/dist
[Info  - 2:23:25 PM] Loading configuration file at /home/zsilversmith/dev/git/repo_root/pyrightconfig.json
[Warn  - 2:23:26 PM] Python version 2.7 from interpreter is unsupported
[Info  - 2:23:26 PM] Assuming Python platform Linux
[Info  - 2:23:26 PM] No include entries specified; assuming /home/zsilversmith/dev/git/repo_root
[Info  - 2:23:26 PM] Setting pythonPath for service "repo_root": "/usr/bin/python"
Search paths found for configured python interpreter:
...
[Warn  - 2:23:26 PM] stubPath /home/zsilversmith/dev/git/repo_root/typings is not a valid directory.
[Warn  - 2:23:26 PM] Python version 2.7 from interpreter is unsupported
[Info  - 2:23:26 PM] Searching for source files
[Warn  - 2:23:30 PM] Skipping recursive symlink "/home/zsilversmith/dev/git/repo_root/web/frontend/node_modules/node_modules" -> "/home/zsilversmith/dev/git/repo_root/web/frontend/node_modules"

pyrightconfig.json (located in git root):

{
    "exclude": [
        ...
        "**/home/zsilversmith/dev/git/repo_root/web/frontend/node_modules**",
    ]
}```

[erictraut]

The glob pattern you're using in the "exclude" is incorrect. The "**" wildcard means "one or more folders", but you're using it at the beginning of the path and at the end. I think what you mean is "**/node_modules". Please give that a try and let us know if it fixes your problem.

[zbs]

Ok, that seems to have gotten it working again, albeit a little more slowly seemingly than 2021.3.1. Thanks for the help!