-
-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
attribute error due to circular import #40584
Comments
This bug applied to 2.3 and 2.4. It probably applies to Under some circumstances, code like: import eek.foo.baz
y = eek.foo.baz.y fails with an attribute error for "foo" if foo is still I've attached a zip file of a demo package "eek" that import eek.foo you'll get the attribute error described above. I think the problem is that eek's foo attribute isn't |
Logged In: YES Are the semantics of import clear enough to confirm that |
Logged In: YES If the semantics aren't clear, then isn't that a bug in A simple rule would seem to be "a package should not try to Given the nature of __all__ and __path__ for packages, I |
Logged In: YES Nick, the semantics of circular imports aren't clear even if no While I expect this bug report to sit here for years (it's hard import baz
y = baz.y instead of with import eek.foo.baz
y = eek.foo.baz.y Then it stops referencing attributes that don't exist before |
Logged In: YES Lowering the priority, as this apparently is not a |
I think the lowered priority got lost somewhere along the line. |
I have a very similar issue (maybe the same?) at the moment. Assume the follwing package structure: main.py main.py says: from package import moduleX moduleX.py says: from . import moduleY and moduleY.py says: from . import moduleX However, this doesn't work: bronger@wilson:~/temp/packages-test$ python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
from package import moduleX
File "/home/bronger/temp/packages-test/package/moduleX.py", line
1, in <module>
from . import moduleY
File "/home/bronger/temp/packages-test/package/moduleY.py", line
1, in <module>
from . import moduleX
ImportError: cannot import name moduleX If I turn the relative imports to absolutes ones, it works. But I'd If you split a large module into chunks, cyclic imports are hardly Note that "import absolute.path.to.module as short" doesn't work either. |
This is actually a pretty tough problem - fixing it would involve some However, I'll give a detailed description of the problem and a possible At the moment, when resolving an import chain __import__ only sets the The most likely solution to the problem would be to change the attribute The consequence of this would be that circular imports would be A clean failure indicating "You have a circular import, get rid of it" |
I dare to make a follow-up although I have no idea at all about the First, I think that circular imports don't necessarily exhibit a So, while I think that this clean error message Nick suggests is a good And secondly, the problem with modules that are not yet populated with |
This came up on python-dev again recently: |
Good sleuthing Nick! It's clearly the same bug that Fredrik found. I tried to test if using Brett' importlib has the same problem, but it I believe that at some point many of the details of importlib should be |
Sorry, never mind about the importlib bug, that was my mistake. I conclude that this is probably the best semantics of import that we I propose to close this as "works as intended" -- and perhaps document |
Maybe it's better to leave it open, waiting for someone to pick it up, In my opinion, this is suprising behaviour without an actual rationale, What bothers me most is that from . import moduleX doesn't work but import package.moduleX does work. So the circular import itself works without problems, |
I just had a thought: we may be able to eliminate this behaviour without What if the import semantics were adjusted so that, as a last gasp Not a fully fleshed out idea at this point (and possibly symptomatic of |
I'm sorely tempted to apply the Van Lindberg clause to the last two |
No argument from me that my suggestion is a mere glimmering of an idea, It was just an angle of attack I hadn't seen suggested before, so I Figuring out a way to set the attribute in the parent package and then |
The key distinction between this and a "bad" circular import is that An ugly fix could be done today for module imports by creating a proxy |
I have done a lazy importer like you describe, Adam, and it does help |
It'd probably be sufficient if we raised "NameError: lazy import 'foo' |
Changed the issue title to state clearly that the core issue is with circular imports that attempt to reference module contents at import time, regardless of the syntactic form used. All of the following module level code can fail due to this problem: from . import a
from package import submodule
from module import a
import module
module.a |
In Torsten's example from . import moduleX can be replaced with moduleX = importlib.import_module('.moduleX', __package__) (*) or moduleX = importlib.import_module('package.moduleX') If that is not pretty enough then perhaps the new syntax import .moduleX could be introduced and made equivalent to (*). |
The implementation of issue bpo-17636 (making IMPORT_FROM fall back to sys.modules when appropriate) will make "import x.y" and "from x import y" equivalent for resolution purposes during import. That covers off the subset of circular references that we want to allow, so I'm closing this one in favour of the more precisely defined proposal. |
I'm reopening this, since PEP-451 opens up new options for dealing with it (at least for loaders that export the PEP-451 APIs rather than only the legacy loader API, which now includes all the standard loaders other than the ones for builtins and extension modules) The attached patch doesn't have an automated test and is quite rough around the edges (as the additional check for the parent being in sys.modules confuses a couple of the importlib tests), but it proves the concept by making the following work: [ncoghlan@lancre py3k]$ cd ../play
[ncoghlan@lancre play]$ mkdir issue992389
[ncoghlan@lancre play]$ cat > issue992389/mod.py
from . import mod
print("Success!")
[ncoghlan@lancre play]$ python3 -c "import issue992389.mod"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "./issue992389/mod.py", line 1, in <module>
from . import mod
ImportError: cannot import name mod
[ncoghlan@lancre play]$ ../py3k/python -c "import issue992389.mod"
Success! |
The new patch will have weird results in the case of a parent module that defines an attribute that's later replaced by an import, e.g. if foo/init.py defines a variable 'bar' that's a proxy for the foo.bar module. This is especially problematic if this proxy is used during the process of importing foo.bar At the very least, this code should NOT be deleting the original foo.bar attribute, but rather restoring its previous value. All in all, I don't think this is a productive route to take. It was discussed on Python-dev previously and IIRC I outlined all the other reasons why back then. The approach in bpo-17636 is the only one that doesn't change the semantics of any existing, not-currently-broken code. In contrast, the proposed change here introduces new side-effects and *more* volatile state and temporal coupling. I don't think this should go in, since the other approach *only* affects execution paths that would currently raise ImportError. |
Belatedly agreeing with PJE on this one. If concerns around circular imports continue to be raised in a post Python 3.5 world (with the bpo-17636 change), then we can look at revisiting this again, but in the meantime, lets just go with the sys.modules fallback. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: