Skip to content

Conversation

andrew-christianson
Copy link
Contributor

In some situations, pyls can call Document.sys_path very frequently. That
method calls jedi.api.environment.get_default_environment(), which can call
jedi.api.environment.find_system_pythons(). On systems with pyenv and many
python versions installed, this can be expensive, as jedi._compatibilty.which
returns a shim path for python major version. Each found version spawns a
subprocess to check the specific version of the python found at the path (via
creation of a jedi Environment). However, if it's only found the pyenv shim,
when a subprocess is spawned, the shim fails to find the version, and pyenv
spends significant time looking for the requested command in other installed
pythons.

Calling .get_cached_default_environment insures jedi goes through the above
process no more than once per 10 minutes per instance of pyls.

Even on systems without pyenv, calling get_default_environment() directly
results in at least one subprocess invocation, and a new jedi Environment object
for each invocation. As Document.sys_path can be called as often as once per
keypress (depending on the client, though this seems to be the case for
lsp-mode) reducing work on that path provides noticeable improvement in
performance.

In some situations, pyls can call Document.sys_path _very_ frequently. That
method calls jedi.api.environment.get_default_environment(), which can call
jedi.api.environment.find_system_pythons(). On systems with pyenv and many
python versions installed, this can be expensive, as jedi._compatibilty.which
returns a shim path for python major version. Each found version spawns a
subprocess to check the specific version of the python found at the path (via
creation of a jedi Environment). However, if it's only found the pyenv shim,
when a subprocess is spawned, the shim fails to find the version, and pyenv
spends significant time looking for the requested command in other installed
pythons.

Calling .get_cached_default_environment insures jedi goes through the above
process no more than once per 10 minutes per instance of pyls.

Even on systems without pyenv, calling get_default_environment() directly
results in at least one subprocess invocation, and a new jedi Environment object
for each invocation. As Document.sys_path can be called as often as once per
keypress (depending on the client, though this seems to be the case for
lsp-mode) reducing work on that path provides noticeable improvement in
performance.
@palantirtech
Copy link
Member

Thanks for your interest in palantir/python-language-server, @andrew-christianson! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

Copy link
Contributor

@gatesn gatesn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrew-christianson thank you very much for this, it's a good catch.

@gatesn gatesn merged commit 63e3cad into palantir:develop Aug 26, 2018
jsenin pushed a commit to jsenin/python-language-server that referenced this pull request Dec 18, 2018
In some situations, pyls can call Document.sys_path _very_ frequently. That
method calls jedi.api.environment.get_default_environment(), which can call
jedi.api.environment.find_system_pythons(). On systems with pyenv and many
python versions installed, this can be expensive, as jedi._compatibilty.which
returns a shim path for python major version. Each found version spawns a
subprocess to check the specific version of the python found at the path (via
creation of a jedi Environment). However, if it's only found the pyenv shim,
when a subprocess is spawned, the shim fails to find the version, and pyenv
spends significant time looking for the requested command in other installed
pythons.

Calling .get_cached_default_environment insures jedi goes through the above
process no more than once per 10 minutes per instance of pyls.

Even on systems without pyenv, calling get_default_environment() directly
results in at least one subprocess invocation, and a new jedi Environment object
for each invocation. As Document.sys_path can be called as often as once per
keypress (depending on the client, though this seems to be the case for
lsp-mode) reducing work on that path provides noticeable improvement in
performance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants