import validator "can't import" when it should have succeeded #488

jt-wang · 2016-06-07T16:35:22Z

The import validator gives an error for the following line:

from lists.views import home_page

The import validator fails to recognize the function/class/variable that I declare time to time. Sometimes it works and successfully in validation, but most of the time it fails and gives an error saying "can't import home_page".

The text was updated successfully, but these errors were encountered:

jt-wang · 2016-06-07T16:47:55Z

Besides, I found a strange thing:

If I comment out the first two lines of views.py,

# from django.shortcuts import render
# from django.http import HttpResponse

def home_page(request):
    return HttpResponse('<html><title>To-Do lists</title></html>')

the import validator works perfectly (though the codes are wrong) and can detect the functionhome_page.

If I add the first line back from django.shortcuts import render,

from django.shortcuts import render
# from django.http import HttpResponse

def home_page(request):
    return HttpResponse('<html><title>To-Do lists</title></html>')

then the validator fails. It can only detect render.

It I comment out the first line and add the second line back from django.shortcuts import HttpResponse,

from django.shortcuts import render
# from django.http import HttpResponse

def home_page(request):
    return HttpResponse('<html><title>To-Do lists</title></html>')

then the validator fails. It can only detect HttpResponse.

If I add the first two lines back,

from django.shortcuts import render
from django.http import HttpResponse

def home_page(request):
    return HttpResponse('<html><title>To-Do lists</title></html>')

the validator fails. It can only detect HttpResponse, too.

That's quite weird.

jt-wang · 2016-06-07T16:54:16Z

FYI:

I checked the project I wrote before, which absolutely went smoothly using Anaconda and I haven't made any changes in it. To my surprise, Anaconda failed in the same way in that project, too.

jt-wang · 2016-06-07T17:33:35Z

To summarize the problem:

import validator failed to detect user-adding things including class, functions, variables and so on as long as there exists from a import b before in the file.

DamnWidget · 2016-06-08T12:29:11Z

Hi, I will take a look on this as soon as I can.

DamnWidget · 2016-07-09T18:40:51Z

I could not replicate the problem as it has been described in the issue, anyway, I found that sometimes Jedi fails to imports modules in some cases with specific relative imports.

Sadly I am not able to offer a satisfactory solution yet to this problem.

leal-alvaro · 2016-11-07T14:11:21Z

Hey, I managed to consistently replicate the problem, but I couldn't fix it.
I created a 2 min video of it and also attached my example files.

(Sorry for GIF quality and size... GitHub doesn't accept MKV video files. Click this image to see it full size, other wise you won't be able to read my comments.).

Here are the source files:
anaconda_import_validator_bug.tar.gz

DamnWidget · 2016-11-07T14:31:34Z

Interesting. Did you noticed if there was any exceptions being logged into your logs while you were recording the video?

leal-alvaro · 2016-11-07T14:36:59Z

I just tested again while looking at the console... There's no exception, error or anything out of the ordinary. Do I have to be in debug mode to see it?

DamnWidget · 2016-11-07T14:54:48Z

I mean this logs @Effenberg0x0 https://github.com/DamnWidget/anaconda#anaconda-behaves-slowly-and-jsonserver-crashes

leal-alvaro · 2016-11-07T15:05:45Z

Oh, sorry. Just tailed it. All I see are INFO logging msgs, no errors or exceptions of any kind. When I replicate the exact thing you see in the GIF above, I get apparently normal "client requests: autocomplete" and the answers (completion options) match what we see in screen in the GIF.

leal-alvaro · 2016-11-07T16:49:37Z

I can confirm that jedi returns the proper completions when the import statement in my_package.my_module is either commented or not commented.

Python 3.5.2+ (default, Sep 22 2016, 12:18:14) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.


In [1]: # The import statement in my_package.my_module is NOT commented right now
      2 import jedi
      3 source="".join(open("bug_main.py", 'r').readlines())

In [2]: jedi.Script(source_path=".", source=source, line=36, column=14).completions()
Out[2]: [<Completion: my_mod>]

In [3]: # Now it should show valid completions for "my_mod" object on the next column.
      2 jedi.Script(source_path=".", source=source, line=36, column=15).completions()
Out[3]: 
[<Completion: description>,
 <Completion: do_that_other_stuff>,
 <Completion: do_that_stuff>,
 <Completion: name>,
 <Completion: __class__>,
 <Completion: __delattr__>,
 <Completion: __dir__>,
 <Completion: __doc__>,
 <Completion: __eq__>,
 <Completion: __format__>,
 <Completion: __ge__>,
 <Completion: __getattribute__>,
 <Completion: __gt__>,
 <Completion: __hash__>,
 <Completion: __init__>,
 <Completion: __le__>,
 <Completion: __lt__>,
 <Completion: __ne__>,
 <Completion: __new__>,
 <Completion: __reduce__>,
 <Completion: __reduce_ex__>,
 <Completion: __repr__>,
 <Completion: __setattr__>,
 <Completion: __sizeof__>,
 <Completion: __str__>,
 <Completion: __subclasshook__>]

leal-alvaro · 2016-11-07T18:11:57Z

I found another interesting thing, still using the example I posted above.
With "import logging" not commented in my_package.my_module, pressing '.' after my_mod gives wrong completions as expected. But if I kill -s HUP the PID of jsonserver.py, I get correct completions for about 2 seconds. This can be repeated indefinitely.

In other words, completion works properly for a couple seconds after jsonserver.py is started.

DamnWidget · 2016-11-07T19:45:07Z

What happens if you disable the import validations and just autocomplete that module?

leal-alvaro · 2016-11-07T19:53:18Z

Same exact behavior: No proper completions unless I comment out the import statement in the imported module.

DamnWidget · 2016-11-08T14:51:55Z

To add more information into this. if you start ST3 with validate_imports set as false the behaviour is 100% correct until you enable the validate_imports option that it fails and stays in that state until ST3 is restarted again.

This is a weird one.

leal-alvaro · 2016-11-08T15:42:40Z

I was wondering, it's kind of a big one. Shouldn't we be seeing more people complaining about this? Maybe it doesn't affect all platforms? I'm testing on Linux only.

DamnWidget · 2016-11-08T15:50:33Z

I will check in OS X and Windows later in home

DamnWidget · 2016-11-08T17:46:28Z

I can confirm that behaviour is consistently reproducible in both OS X and Windows 10

leal-alvaro · 2016-11-08T17:53:56Z

Ok. I'm not really familiar with the code base, but I'll try to understand
the bug and hopefully come up with a fix in the next days.

On Tue, Nov 8, 2016, 15:46 Oscar Campos notifications@github.com wrote:

I can confirm that behaviour is consistently reproducible in both OS X and
Windows 10

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#488 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD1KjviDHKxudEsv6bmXFuY4PFQMal0Eks5q8LV1gaJpZM4IwHrJ
.

leal-alvaro · 2016-11-09T21:19:32Z

I rolled back each commit and tested with the python source files I posted here to see when the bug was introduced.

This is very weird but I have verified that the bug was actually introduced by commit dd960dc ("anaconda JsonServer totally decoupled from anaconda ST3 plugin (contained on anaconda still)"), back in August 2014! I have no idea how no one (including me) noticed it until now.

Anyway, the problem now is that this is a huge commit, there's a lot to learn and a lot to check. If I had to take a wild guess: I noticed listeners.linting.BackgroundLinter() methods make repeated calls to anaconda_lib.helpers.check_linting(), and if I comment out all those if statements and just make it return True, it seems to kinda improve it. Maybe something is wrong with those bitwise AND flags logic?

@DamnWidget, any thoughts? Am I close to it? :)

DamnWidget · 2016-11-09T22:45:01Z

Hard to say Alvaro, I am still struggling myself to understand how the linting could affect in any way the auto completion as they are completely decoupled features, one should not affect the other. I have no idea how Jedi could stop giving us results after any linting option is used.

If I was you, I would start adding some logging verbosity into the jedi handler in the anaconda_server so you can see what the Jedi library is doing and how it behaves. For me is a big mystery how this could be happening, unfortunately I don't have many time now and I have lot of work to do.

I hope you can find a solution, I will provide you all the guidance that I could.

DamnWidget · 2016-11-23T15:15:46Z

Guys, this totally looks like a bug in jedi, you can confirm it following the steps below in an IPython or Python REPL and the files that @Effenberg0x0 provided (here) :

a = 'from my_package.my_module import MyModule'
b = 'import logging'

import jedi
print(jedi.Script(a, 1, 37, '<your_own_path_to_files>/bug_main.py').goto_assignments())
print(jedi.Script(b, 1, 10, '<your_own_path_to_files>/my_package/my_module.py').goto_assignments())

Until here everything is fun and happy, but if you try to get the assignments for the a again:

print(jedi.Script(a, 1, 37, '<your_own_path_to_files>/bug_main.py').goto_assignments())

You get an empty list.

Actually I "fixed" it in anaconda, and I double quote the word fixed because is just a workaround that uses the Jedi API wrong just not specifying a file path to the Script constructor, doing that must be avoided, I am gonna alert David about this in Jedi itself.

The "fix" will be released later today/

leal-alvaro · 2016-11-23T15:26:32Z

That's great Oscar :) I have been working my way through Anaconda's code base for some days and trying many different things with no luck at all. Thank you!

DamnWidget · 2016-11-23T17:49:45Z

Could you kindly confirm that this issue has been resolved in 2.1.8?

Thank you.

leal-alvaro · 2016-11-23T18:47:19Z

Hi Oscar, there was a tiny typo that caused Anaconda to not be loaded. I submitted a pull request. Other than that, preliminary tests suggest you fixed it.

DamnWidget · 2016-11-23T19:27:19Z

I just merged your PR, how are we looking?

leal-alvaro · 2016-11-23T19:37:29Z

I installed Sublime Text 3 fresh on a new Ubuntu VM and installed Anaconda via Package Control. I think everything is fine :)

DamnWidget · 2016-11-23T20:03:48Z

If @jt-wang is still around. can you please confirm and close?

jt-wang · 2016-11-24T05:38:51Z

@DamnWidget Yes. It is perfect now. Many thanks for your effort @DamnWidget @Effenberg0x0

DamnWidget added bug S: major Jedi labels Nov 8, 2016

DamnWidget mentioned this issue Nov 23, 2016

ImportValidator doesn't like relative imports #432

Closed

DamnWidget added the resolved label Nov 23, 2016

DamnWidget added a commit that referenced this issue Nov 23, 2016

workaround what seems to be a Jedi bug, reported in #488 and #432

d6b1adc

jt-wang closed this as completed Nov 24, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import validator "can't import" when it should have succeeded #488

import validator "can't import" when it should have succeeded #488

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016 •

edited

DamnWidget commented Jun 8, 2016

DamnWidget commented Jul 9, 2016

leal-alvaro commented Nov 7, 2016 •

edited

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 8, 2016

leal-alvaro commented Nov 8, 2016

DamnWidget commented Nov 8, 2016

DamnWidget commented Nov 8, 2016

leal-alvaro commented Nov 8, 2016

leal-alvaro commented Nov 9, 2016

DamnWidget commented Nov 9, 2016 •

edited

DamnWidget commented Nov 23, 2016 •

edited

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

jt-wang commented Nov 24, 2016 •

edited

import validator "can't import" when it should have succeeded #488

import validator "can't import" when it should have succeeded #488

Comments

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016

jt-wang commented Jun 7, 2016 • edited

DamnWidget commented Jun 8, 2016

DamnWidget commented Jul 9, 2016

leal-alvaro commented Nov 7, 2016 • edited

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 7, 2016

leal-alvaro commented Nov 7, 2016

DamnWidget commented Nov 8, 2016

leal-alvaro commented Nov 8, 2016

DamnWidget commented Nov 8, 2016

DamnWidget commented Nov 8, 2016

leal-alvaro commented Nov 8, 2016

leal-alvaro commented Nov 9, 2016

DamnWidget commented Nov 9, 2016 • edited

DamnWidget commented Nov 23, 2016 • edited

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

leal-alvaro commented Nov 23, 2016

DamnWidget commented Nov 23, 2016

jt-wang commented Nov 24, 2016 • edited

jt-wang commented Jun 7, 2016 •

edited

leal-alvaro commented Nov 7, 2016 •

edited

DamnWidget commented Nov 9, 2016 •

edited

DamnWidget commented Nov 23, 2016 •

edited

jt-wang commented Nov 24, 2016 •

edited