Jedi does not care what is in sys.path #36

Closed
tkf opened this Issue Oct 20, 2012 · 7 comments

Comments

Projects
None yet
3 participants
Collaborator

tkf commented Oct 20, 2012

It looks like jedi does not care what is in sys.path when completing code.

Let's say I have a virtualenv ~/.virtualenvs/tmp which has numpy installed and numpy is not installed in system site-packages. If I start jedi using ~/.virtualenvs/tmp/bin/python jedi can complete numpy, but it cannot do it when I start jedi using /usr/bin/python and add numpy to sys.path.

This is how to reproduce. Save the following code in test.py:

import os
import sys
import jedi


def add_virtualenv_path():
    """Add virtualenv's site-packages to `sys.path`."""
    venv = os.path.abspath(os.getenv('VIRTUAL_ENV'))
    if not venv:
        return
    path = os.path.join(
        venv, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages')
    sys.path.insert(0, path)


add_virtualenv_path()
import numpy  # this does not fail

source = """\
import numpy
numpy."""
script = jedi.Script(source, 2, len('numpy.'), '')

print script.complete()

Run test.py using /usr/bin/python, like this:

source ~/.virtualenvs/tmp/bin/activate
echo $VIRTUAL_ENV
/usr/bin/python test.py

This will output:

/home/takafumi/.virtualenvs/tmp
[]

If I run this script by python test.py, I get a long list of completions.

Owner

davidhalter commented Oct 20, 2012

I explain you how it currently works:
https://github.com/davidhalter/jedi/blob/master/jedi/builtin.py#L16

takes the sys path at startup and saves it. All further uses of Jedi depend on that line.

What do you think? Should we change that? I don't know why anyone would use it...
Basically virtualenv support should be something that Jedi does and not some plugin.

Collaborator

tkf commented Oct 20, 2012

I am using the code above in my Emacs plugin and wonder why it does not work. If Jedi provides API to add module path I think that's fine. Why do you need to cache sys.path anyway? Because you need to cache the result of analyzing modules?

BTW if you want to filter out an empty string from sys.path I think it's better to use filter(None, sys.path) as the empty string might be anywhere in the list.

dbrgn referenced this issue in davidhalter/jedi-vim Oct 28, 2012

Closed

Add virtualenv support #31

Collaborator

dbrgn commented Nov 13, 2012

In jedi-vim, the problem is that the interpreter used by vim is not using the virtualenv at all. So you can't detect the virtualenv from "within" the Python code, as it's not in PYTHONPATH used by vim.

If you want to provide virtualenv support from jedi itself, there might be different possibilities:

  • Generic approach: Provide an easy way to add additional PYTHONPATH parts to a completion object
  • Specialized approach: Provide a virtualenv=<path> parameter
  • Rely on env variables (see code at davidhalter/jedi-vim#31 (comment))

I think the plugins should be responsable for PYTHONPATH magic (as long as they don't use a separate python environment it already works as is), but jedi should provide tools to easily detect and/or register virtualenvs.

Collaborator

tkf commented Nov 13, 2012

I am running this function before importing jedi. Doesn't this work in vim, provided that you start vim from a shell in which virtualenv is activated?

def add_virtualenv_path():
    """Add virtualenv's site-packages to `sys.path`."""
    venv = os.getenv('VIRTUAL_ENV')
    if not venv:
        return
    venv = os.path.abspath(venv)
    path = os.path.join(
        venv, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages')
    sys.path.insert(0, path)

https://github.com/tkf/emacs-jedi/blob/master/jediepcserver.py

Collaborator

dbrgn commented Nov 13, 2012

I guess that fix is the best solution we'll be able to come up with, but it could be moved into jedi itself as a helper function. What do you think? @tkf @davidhalter @strycore

Owner

davidhalter commented Nov 13, 2012

I wouldn't use it as a helper function but as the default importing behavior, which means adding this to the cached builtin.module_find_path. (Maybe getting rid of the module_find_path would also be an option if we're changing things).

Probably we should also add an option to settings.py, which could turn this behavior off.

Collaborator

tkf commented Nov 13, 2012

I agree with you. Having the same function in vim/emacs/sublime text bindings does not make sense.

But at same time, I think jedi should expose an API to set sys.path. I am doing lazy import in Emacs binding and because of that, I can't use jedi against jedi functions, which is kind of sad. It's not urgent as we have a workaround, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment