Recursive or looping symlinks can cause language server to repeatedly analyze files

[richardwb]

I commented previously on #2297 but I found that symlinks were still causing problems after I grabbed a recent build today.

---

Environment data

• VS Code version: 1.27.2
• Extension version (available under the Extensions sidebar): 2018.9.0-alpha, retrieved at Sep 18, 10:30AM PDT.
• OS and version: Mac OS 10.13.6
• Python version (& distribution if applicable, e.g. Anaconda): 2.7.14
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A

Actual behavior

The Python language server gets caught in a super deep nest of recursive symlinks. The repro case given completes quickly (there's nothing to analyze) but still demonstrates the problem, I think.

If there is actual code to parse/analyze it pretty much goes on forever. "Analyzing workspace, #### items remaining" goes down and then quickly shoots up, and this repeats over and over.

I can workaround this by adding the offending symlinks/folder structures to a VSCode exclusion list.

Expected behavior

The Python language server should be able to handle these structures in some reasonable way.

Steps to reproduce:

1. Create a folder structure like this

/code
  /test
    /some
      test.py
      /folder
        /symlink -> ../../some
        __init__.py (empty)
        lib.py (empty)

2. test.py only needs to contain import folder.lib.
3. Enable trace output for the language server and watch the logs, it gets progressively deeper and deeper into a repeated folder/symlink/folder/symlink/ folder structure.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Initializing for /usr/local/opt/python/bin/python2.7
Loading files from /Users/username/code/test/some
Parsing document file:///Users/username/code/test/some/test.py
Parsing document file:///Users/username/code/test/some/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/test.py at version -1
Parse complete for file:///Users/username/code/test/some/folder/__init__.py at version -1
Analysis queued for file:///Users/username/code/test/some/test.py
Analysis queued for file:///Users/username/code/test/some/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/lib.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/test.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/test.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/lib.py
Parse complete for file:///Users/username/code/test/some/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/__init__.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
<more>
Waiting for parsing to complete
Parsing complete. Waiting for analysis entries to enqueue
Enqueue complete. Waiting for analysis to complete
Received new analysis for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)

---

Symlinks to a common folder (e.g. two projects which have a symlink to the same folder) also get re-analyzed but that's usually okay as analysis is pretty fast in general.

[ericsnowcurrently]

Thanks for letting us know about this issue, @richardwb. I'm having a little trouble reproducing the issue. I set up a directory structure as described and did not see any problems (nor any unexpected output in the Output panel). I tried with Python 3.7 in a venv, as well as the system 2.7. I also tried the same using Jedi (with no problems). Are there any other details of the setup that would help?

[richardwb]

It could be more obvious with a bit more code to analyze, but turning on trace level output seems to reliably demonstrate this for me in the output window.

I just tried with a clean VSCode Insiders build, with just the Python extension installed (also Insiders build, today's). The only contents of my settings.json file is this:

{
    "python.analysis.logLevel": "Trace",
    "python.jediEnabled": false
}

I took the contents of this archive: test.tar.gz, extracted it to a location (say ~/), and then opened the ~/test folder in VSCode, then browsed to the test.py file with the explorer pane, and I see the same behavior as before.

The version of Python I use doesn't really seem to make a difference, I tried both a pyenv Python 3.6 and a brew-installed Python 2.7.

[ericsnowcurrently]

Thanks for the extra info. I'll take a look today.

[ericsnowcurrently]

Sorry for the delay. I was able to reproduce the problem using the latest extension code (and latest language server). We'll work on sorting it out.

[DonJayamanne]

Created upstream issue microsoft/python-language-server#181

[jclinton]

Upstream issue has been marked fixed.

[DonJayamanne]

Closing as upstream issue has been resolved.