Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Basic Python 3.3 compatibility #109

Closed
wants to merge 1 commit into from

3 participants

@1st1

In python 3.3 the implementation of the import system is rewritten from scratch and based on importlib. Also some new functionality was introduced, such as namespace packages (directories in pythonpath without init.py files)

This fix restores most of failing unittests under 3.3, and adds basic support for namespace packages.

This commit is very WIP, and maybe considered as a proof of concept.

@davidhalter davidhalter commented on the diff
jedi/imports.py
@@ -249,7 +257,24 @@ def follow_str(ns_path, string):
for i, s in enumerate(self.import_path):
try:
current_namespace = follow_str(current_namespace[1], s)
- except ImportError:
+ if not current_namespace[1] and is_py33:
+ loader = importlib.find_loader(s)
@davidhalter Owner

Are you sure this is ever used? Because follow_str tends to or fail with an ImportError. But since I'm not really getting the Python 3.3 imports, could you explain what this part does?

@1st1
1st1 added a note

Yes, it's used under 3.3 when you're inspecting an import of a namespace package. In this case, imp.find_module simply fails with ImportError, however import_lib.find_loader returns _frozen_importlib.NamespaceLoader object.

@1st1
1st1 added a note

Ideally, imports.py needs to be rewritten just for 3.3 (with the use of importlib only), but that's too much of a time investment right now.

@davidhalter Owner

Hmm why don't we use a completely different mechanism for >= py3.3? I think this is not too complicated, because we would just need to rewrite the parts of _follow_file_system until f = None. -> 80 lines.

I have no idea how good importlib is... But it really sounds like we should switch away from imp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@davidhalter davidhalter commented on the diff
jedi/imports.py
@@ -249,7 +257,24 @@ def follow_str(ns_path, string):
for i, s in enumerate(self.import_path):
try:
current_namespace = follow_str(current_namespace[1], s)
- except ImportError:
+ if not current_namespace[1] and is_py33:
+ loader = importlib.find_loader(s)
+ if loader.__name__ == 'BuiltinImporter':
+ current_namespace = (current_namespace[0], s,
+ current_namespace[2])
+
+ except ImportError as ex:
@davidhalter Owner

Could you remove the as ex? Python 2.5 compatibility.

@1st1
1st1 added a note

Sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@1st1 1st1 commented on the diff
jedi/imports.py
@@ -249,7 +257,24 @@ def follow_str(ns_path, string):
for i, s in enumerate(self.import_path):
try:
current_namespace = follow_str(current_namespace[1], s)
- except ImportError:
+ if not current_namespace[1] and is_py33:
+ loader = importlib.find_loader(s)
+ if loader.__name__ == 'BuiltinImporter':
@1st1
1st1 added a note

I don't like this line, I'll take a look if it's possible to change it to isinstance check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@1st1 1st1 commented on the diff
jedi/imports.py
@@ -249,7 +257,24 @@ def follow_str(ns_path, string):
for i, s in enumerate(self.import_path):
try:
current_namespace = follow_str(current_namespace[1], s)
- except ImportError:
+ if not current_namespace[1] and is_py33:
+ loader = importlib.find_loader(s)
+ if loader.__name__ == 'BuiltinImporter':
+ current_namespace = (current_namespace[0], s,
+ current_namespace[2])
+
+ except ImportError as ex:
+ if not i and is_py33:
+ loader = importlib.find_loader(s)
+ if 'NamespaceLoader' in repr(loader):
@1st1
1st1 added a note

'NamespaceLoader' in repr(loader) is also ugly as hell. The problem is that parts of importlib is compiled in a frozen module, and those classes are separated from what we have defined in importlib._bootstrap. Anyways, I'll take a look at this again today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@schlamar

This could be closed now, too :)

@davidhalter
Owner

Closing, because this has been solved by #187.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 28 additions and 2 deletions.
  1. +1 −0  jedi/_compatibility.py
  2. +27 −2 jedi/imports.py
View
1  jedi/_compatibility.py
@@ -5,6 +5,7 @@
import sys
is_py3k = sys.hexversion >= 0x03000000
+is_py33 = sys.hexversion >= 0x03030000
is_py25 = sys.hexversion < 0x02060000
View
29 jedi/imports.py
@@ -13,6 +13,11 @@
import itertools
import cache
+from _compatibility import is_py33
+if is_py33:
+ import importlib
+
+
# for debugging purposes only
imports_processed = 0
@@ -220,10 +225,13 @@ def follow_str(ns_path, string):
elif self.import_stmt.relative_count:
path = self.get_relative_path()
+ if path and not isinstance(path, list):
+ path = [path]
+
global imports_processed
imports_processed += 1
if path is not None:
- return imp.find_module(string, [path])
+ return imp.find_module(string, path)
else:
debug.dbg('search_module', string, self.file_path)
# Override the sys.path. It works only good that way.
@@ -249,7 +257,24 @@ def follow_str(ns_path, string):
for i, s in enumerate(self.import_path):
try:
current_namespace = follow_str(current_namespace[1], s)
- except ImportError:
+ if not current_namespace[1] and is_py33:
+ loader = importlib.find_loader(s)
@davidhalter Owner

Are you sure this is ever used? Because follow_str tends to or fail with an ImportError. But since I'm not really getting the Python 3.3 imports, could you explain what this part does?

@1st1
1st1 added a note

Yes, it's used under 3.3 when you're inspecting an import of a namespace package. In this case, imp.find_module simply fails with ImportError, however import_lib.find_loader returns _frozen_importlib.NamespaceLoader object.

@1st1
1st1 added a note

Ideally, imports.py needs to be rewritten just for 3.3 (with the use of importlib only), but that's too much of a time investment right now.

@davidhalter Owner

Hmm why don't we use a completely different mechanism for >= py3.3? I think this is not too complicated, because we would just need to rewrite the parts of _follow_file_system until f = None. -> 80 lines.

I have no idea how good importlib is... But it really sounds like we should switch away from imp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ if loader.__name__ == 'BuiltinImporter':
@1st1
1st1 added a note

I don't like this line, I'll take a look if it's possible to change it to isinstance check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ current_namespace = (current_namespace[0], s,
+ current_namespace[2])
+
+ except ImportError as ex:
@davidhalter Owner

Could you remove the as ex? Python 2.5 compatibility.

@1st1
1st1 added a note

Sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ if not i and is_py33:
+ loader = importlib.find_loader(s)
+ if 'NamespaceLoader' in repr(loader):
@1st1
1st1 added a note

'NamespaceLoader' in repr(loader) is also ugly as hell. The problem is that parts of importlib is compiled in a frozen module, and those classes are separated from what we have defined in importlib._bootstrap. Anyways, I'll take a look at this again today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ try:
+ m = __import__(s)
+ except ImportError:
+ pass
+ else:
+ current_namespace = (None, list(m.__path__), None)
+ continue
+
if self.import_stmt.relative_count \
and len(self.import_path) == 1:
# follow `from . import some_variable`
Something went wrong with that request. Please try again.