Intellisense speedup proposals

[piotr-rarus]

Environment data

• VS Code version: 1.28.2

• Extension version (available under the Extensions sidebar): 2018.9.1 / issues observed since May/June 2018

• OS and version: Windows 10, latest update

• Python version (& distribution if applicable, e.g. Anaconda): plain Python 3.7.0 64-bit

• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv

• Relevant/affected Python packages and their versions:

  • atomicwrites==1.2.1
  • attrs==18.2.0
  • colorama==0.4.0
  • cycler==0.10.0
  • flake8==3.5.0
  • imbalanced-learn==0.4.1
  • kiwisolver==1.0.1
  • matplotlib==3.0.0
  • mccabe==0.6.1
  • more-itertools==4.3.0
  • numpy==1.15.2
  • pandas==0.23.4
  • pluggy==0.7.1
  • progressbar2==3.38.0
  • py==1.6.0
  • pycodestyle==2.3.1
  • pyflakes==1.6.0
  • pyparsing==2.2.2
  • pytest==3.8.2
  • python-dateutil==2.7.3
  • python-utils==2.3.0
  • pytz==2018.5
  • scikit-learn==0.20.0
  • scipy==1.1.0
  • seaborn==0.9.0
  • six==1.11.0

Actual behavior

Intellisense performance downgrade. It takes around 30s to search for particular docstring. IDE just display 'Loading...' window frame.

Expected behavior

Intellisense completion to be super-duper fast.

Steps to reproduce:

Use many, heavy dependencies import statements inside a module.
i.e.

from sklearn.impute import SimpleImputer
from imblearn import under_sampling, over_sampling, combine
from sklearn import preprocessing
from sklearn.decomposition import PCA
from sklearn.model_selection import RepeatedStratifiedKFold
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import classification_report

Logs

Outside interpreter runtime.

Message

###Dear developers, contributors, users

I fell in love with VSCode, but recent Intellisense performance drop, puts my feeling to test.
I don't know much about Intellisense implementation, but my guess is, that it does lazy evaluation of
docstrings in imported modules, when displaying available prompts. Certainly this improves
memory consumption. Personally, I use VSCode for ML/CV related projects in Python.
Python is by far, most reasonable tool for such field, because there're a lot of great libs.
Downside is that my whole lib stack (NumPy, SciPy, Pandas, Scikit-Learn, Scikit-Image,, OpenCV, Imbalanced-Learn, Matplotlib, Seaborn, Keras, Tensorflow, PyTorch, etc, and much more of their dependencies), is quite heavy. Don't know much about other languages/implementations, but in CPython, it takes few seconds to startup a machine and load up dependencies before execution.

As developers, we mostly work on decent machines (16GB at least, I think 32/64GB is standard for ML/CV applications). For me, it wouldn't be much of expense (just a few GB of RAM) to load doc-strings from all project dependencies eagerly, and pre-cache them. Certainly, startup will take few minutes, but I can use that time to make a coffee (which I eventually always do, when I start my work). I'd rather waste few minutes for startup (which is one-time expense), and rejoice later upon Intellisense great performance, than pulling my hair out, during whole day of work.

###PS
To be more exact: we're observing 15-30 second lag, which makes Intellisense practically unusable.

###PS2
to keep startup time in place, we could only pre-cache docstrings from opened files.
Import statements could be traversed one by one, and docstrings collected.
Imported dependencies should be tracked to re-evaluate docstrings cache.
This would keep RAM usage in place too.

Final performance would be also CPU/RAM efficient, as docstring should be kept in dictionary-like collection. Dictionaries are sweet and fast, and we would only keep docstrings as values (strings aren't that much heavy). When prompting Intellisense, it would fetch docstring from cache and render it.
Accessing values in dictionary is super-super-super-fast, and rendering wouldn't also be a problem

Cache should be shared among opened files/modules.

###PS3
To make it even more efficient:
when considering interpreted languages i.e. Python, JS, etc., we don't even need to import those dependencies. They're not compiled/pre-compiled. We could just as easily load them as text files, and fetch docstrings using simple regex.

Tracking dependencies wouldn't also be much of a problem. It could simply be done using regex and trivial parsing.

###PS4
I apologize, if you find my attitude to be demanding, and leech in nature.
I'd be happy to help to contribute, but I don't hold much of experience in both front-end and IDE development.

Kind Regards,
Piotr Rarus

[montant]

I'm facing the very same issue, and I have to disable Python extension for editing... python files because of it. I have no opinion on the way to go for handling it but this issue is impacting usability drastically for sure.

[piotr-rarus]

I am aware of python.autoComplete.extraPaths, python.autoComplete.preloadModules, but both doesn't seem to provide satisfactory results.
It would also be cumbersome to speficy modules by hand, to pre-load them for each workspace.

[brettcannon]

Please try "python.jediEnabled": false as that will turn on our new language server which is faster than Jedi.

[piotr-rarus]

Thanks, I love you <3. It works much smoother now.

But here are another issues:

I'm getting some errors from your linter, when I try to import my own modules (though there's no problem resolving during runtime).

Due to this error, docstrings from my own modules aren't fetched, and I'm still running blind.
Only docstrings from currently opened module are fetched.

I guess that's because of my project structure:

I know that grouping code under ./src/ is ill advised, but I like to keep my code separate from data/env/logs.

It would be nice to let user configure project code root folder for Intellisense.

One more little thing: strangely I get this sensation, that docstrings render doesn't look that nice, like the previous one.

"python.jediEnabled": false

python.jediEnabled": true

Would it also be possible to display parameters list in each separate line, if they don't fit?

[piotr-rarus]

Eventually I did:

"python.jediEnabled": true,
"python.jediMemoryLimit": 4096,

It helped.

I suspect, that extension developers, got some guidelines from Microsoft to keep memory usage low.
After all VSCode aims to be fast and lightweight IDE. From what I've gathered, when observing memory consumption, the default memory limit (1024 MB), isn't set for Intellisense only, but for IDE itself.
This leaves us with ~200MB for Intellisense, which is a little to low. When memory consumption exceeds the cap, something strange happens, and Intellisense performance drops significantly.
Setting the cap higher helps a lot, and memory consumption only growths by ~200 MB at max in my case.

I've traveled a long long way to get to this point. I saw a lot of people complaining about Intellisense performance, and nowhere did I see the trick of ramping up memory cap.
I'd like to make request: could this be temporarily shared between devs in some more visible place?
After all we don't want to mess this repository, by creating another Intellisense performance complaints.
Some FAQ would do.

[piotr-rarus]

Let me make another guess:
Did you wrote vscode-python Intellisense implementation in Python?
This could explain memory consumption issues by Intellisense.
It runs python machine underneath, which is not that lightweight.
Furthermore, when python de-allocates object's memory space, it isn't immediately return to the system. Python machine still holds this memory for further use (I don't why it's that way, but i suspect devs had their reasons.)

Obviously we're talking about CPython implementation, which is most common.

[brettcannon]

The memory limit is because we had several instances of where Jedi was blowing out people's memory due to its caching behaviour. And Jedi is implemented in Python as you surmised.

If you are having issues with the language server, you can test experimental builds by setting "python.analysis.downloadChannel": "daily", otherwise some of your comments above already have open issues. If they don't then feel free to open new ones.