Exception in Jedi crashes ptpython (another nother one)

[liavkoren]

Hi. I'm on the current cheese-shop release. Test.py is:

def foo(dt):
    return True

def bar(usage_records):
    a_lambda = lambda (foo): 'foobar' == 'foobar' 

in a clean virtual env, (only pip installed ptpython and ipython), launching ptpython and then doing:

import test
test.foo(<any character>...

crashes jedi with the following stack trace:

>>> test.foo(aException in thread Thread-142:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/prompt_toolkit/interface.py", line 647, in run                                         
    completions = list(buffer.completer.get_completions(document, complete_event))
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/ptpython/completer.py", line 127, in get_completions
    completions = script.completions()
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/api/__init__.py", line 192, in completions
    for call_sig in self.call_signatures():
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/api/__init__.py", line 521, in call_signatures
    self.source, self._pos)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/cache.py", line 93, in wrapper
    value = next(generator)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/cache.py", line 115, in cache_call_signatures
    yield evaluator.eval_element(call)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/cache.py", line 41, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/__init__.py", line 183, in eval_element
    types = self._eval_atom(element.children[0])
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/__init__.py", line 230, in _eval_atom
    return self.find_types(scope, atom, stmt.start_pos, search_global=True)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/__init__.py", line 120, in find_types
    return f.find(scopes, search_global)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/debug.py", line 52, in wrapper
    result = func(*args, **kwargs)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/finder.py", line 86, in find
    names = self.filter_name(scopes)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/finder.py", line 177, in filter_name
    names = self.names_dict_lookup(names_dict, position)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/finder.py", line 125, in names_dict_lookup
    names = filter_definition_names(names, self.name_str, position)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/finder.py", line 57, in filter_definition_names
    stmt = names[0].get_definition()
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/evaluate/helpers.py", line 159, in get_definition
    return self.parent
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/cache.py", line 143, in wrapper
    result = func(self)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/api/interpreter.py", line 83, in parent
    mod = FastParser(load_grammar(), source, path[:-1]).module
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/fast.py", line 106, in __call__
    p = super(CachedFastParser, self).__call__(grammar, source, module_path)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/fast.py", line 243, in __init__
    self.update(source)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/fast.py", line 257, in update
    self._parse(source)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/fast.py", line 383, in _parse
    line_offset, nodes)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/fast.py", line 435, in _get_node
    p = Parser(self._grammar, parser_code, self.module_path, tokenizer=tokenizer)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/__init__.py", line 155, in __init__
    self.module = p.parse(self._tokenize(tokenizer))
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/pgen2/parse.py", line 106, in parse
    if self.addtoken(type, value, prefix, start_pos):
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/pgen2/parse.py", line 170, in addtoken
    value, start_pos, prefix, self.addtoken)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/__init__.py", line 269, in error_recovery
    self._stack_removal(grammar, stack, index + 1, value, start_pos)
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/__init__.py", line 304, in _stack_removal
    clear_names(children=node[1])
  File "/Users/liavkoren/Envs/test/lib/python2.7/site-packages/jedi/parser/__init__.py", line 298, in clear_names
    self._scope_names_stack[-1][c.value].remove(c)
KeyError: u'a_lambda'

[liavkoren]

Happy to try and help out on this.

[jonathanslenders]

Thanks for reporting this bug! It's something that has to be fixed in the Jedi source code. We will have to create an issue there.
The only thing that we can do in ptpython is just catching this KeyError, and documenting why we did that.

[jonathanslenders]

This should fix it: 217c59f

I plan to push a new release in January.