bpo-37409: fix relative import with no parent #14956

benjimin · 2019-07-26T08:29:50Z

builtins.__import__ was missing a check to flag that if no dot was found in the __name__ of a module that isn't explicitly a submodule of a package then a relative import is incorrect.

https://bugs.python.org/issue37409

…ere no package exists Relative imports use resolve_name to get the absolute target name, which first seeks the current module's absolute package name from the globals: If __package__ (and __spec__.parent) are missing then as a hail mary it uses __name__, truncating the last segment if the module is any submodule rather than a package __init__.py (which it guesses from whether __path__ is defined). The hail mary should fail if there is no parent package (top level modules), if __name__ is uninformatively '__main__' (-m entry points), or both (scripts). That is, if both __name__ has no subcomponents and the module does not seem to be a package __init__ module. (Bug silently used module as if package, aliasing unexpected objects. Note importlib contained an unaffected alternative __import__ implementation.)

benjimin · 2019-07-31T02:24:39Z

This is a conservative (minimal) change, intended to also be appropriate for back-porting.

The bug was that (sibling-level) relative import statements would in some cases (when unable to identify any enclosing package) instead retrieve objects from the current module rather than raising the expected ImportError. (Only builtins.__import__ was affected, not importlib.__import__)

Specifically, the problem concerned circumstances where __package__ and __spec__.parent are uninformative and the current module is not itself a package (i.e. __path__ is undefined, contrary to common __init__.py). In these circumstances either the package can be identified from __name__ (after truncating the last .component), or no package can be found (e.g. top-level modules and, more commonly, scripts/modules run as "__main__"). This last case was failing to trigger (instead treating the module itself as the package).

brettcannon

I have a suggestion on how to make the fix while taking a more explicit/simpler solution.

And please add a news entry for this.

Lib/test/test_builtin.py

Lib/test/test_import/__init__.py

Python/import.c

bedevere-bot · 2019-08-02T22:10:50Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

benjimin · 2019-08-07T14:11:48Z

I have made the requested changes; please review again

bedevere-bot · 2019-08-07T14:11:51Z

Thanks for making the requested changes!

@brettcannon: please review the changes made to this pull request.

brettcannon

Some very minor tweaks and then this will be ready to be merged!

Python/import.c

Misc/NEWS.d/next/Core and Builtins/2019-08-06-23-39-05.bpo-37409.1qwzn2.rst

Lib/test/test_import/__init__.py

bedevere-bot · 2019-08-09T22:12:24Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

benjimin · 2019-08-10T13:32:49Z

@brettcannon is this also appropriate for backporting to 3.7?

I hope this fix may help people encounter more informative error messages when they are first learning/experimenting with explicit relative imports (say, migrating code from py2).

I have made the requested changes; please review again

bedevere-bot · 2019-08-10T13:32:52Z

Thanks for making the requested changes!

@brettcannon: please review the changes made to this pull request.

encukou

This looks great!
@brettcannon, do you still want to re-review?

miss-islington · 2019-09-11T10:09:49Z

Thanks @benjimin for the PR, and @brettcannon for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖

bedevere-bot · 2019-09-11T10:09:49Z

@brettcannon: Please replace # with GH- in the commit message next time. Thanks!

miss-islington · 2019-09-11T10:09:55Z

Sorry, @benjimin and @brettcannon, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 92420b3e679959a7d0ce875875601a4cee45231e 3.8

miss-islington · 2019-09-11T10:09:56Z

Sorry @benjimin and @brettcannon, I had trouble checking out the 3.7 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 92420b3e679959a7d0ce875875601a4cee45231e 3.7

bedevere-bot · 2019-09-11T10:40:55Z

GH-15913 is a backport of this pull request to the 3.8 branch.

) Relative imports use resolve_name to get the absolute target name, which first seeks the current module's absolute package name from the globals: If __package__ (and __spec__.parent) are missing then import uses __name__, truncating the last segment if the module is a submodule rather than a package __init__.py (which it guesses from whether __path__ is defined). The __name__ attempt should fail if there is no parent package (top level modules), if __name__ is '__main__' (-m entry points), or both (scripts). That is, if both __name__ has no subcomponents and the module does not seem to be a package __init__ module then import should fail.. (cherry picked from commit 92420b3) Co-authored-by: Ben Lewis <benjimin@users.noreply.github.com>

…ythonGH-15913) Relative imports use resolve_name to get the absolute target name, which first seeks the current module's absolute package name from the globals: If __package__ (and __spec__.parent) are missing then import uses __name__, truncating the last segment if the module is a submodule rather than a package __init__.py (which it guesses from whether __path__ is defined). The __name__ attempt should fail if there is no parent package (top level modules), if __name__ is '__main__' (-m entry points), or both (scripts). That is, if both __name__ has no subcomponents and the module does not seem to be a package __init__ module then import should fail.. (cherry picked from commit 92420b3) Co-authored-by: Ben Lewis <benjimin@users.noreply.github.com> (cherry picked from commit 0a6693a) Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>

brettcannon · 2019-09-11T11:39:39Z

#15925 is the 3.7 backport

) Relative imports use resolve_name to get the absolute target name, which first seeks the current module's absolute package name from the globals: If __package__ (and __spec__.parent) are missing then import uses __name__, truncating the last segment if the module is a submodule rather than a package __init__.py (which it guesses from whether __path__ is defined). The __name__ attempt should fail if there is no parent package (top level modules), if __name__ is '__main__' (-m entry points), or both (scripts). That is, if both __name__ has no subcomponents and the module does not seem to be a package __init__ module then import should fail.. (cherry picked from commit 92420b3) Co-authored-by: Ben Lewis <benjimin@users.noreply.github.com> (cherry picked from commit 0a6693a) Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>

brettcannon · 2019-09-11T12:50:59Z

And all the backports are done! Thanks, @benjimin .

Relative imports use resolve_name to get the absolute target name, which first seeks the current module's absolute package name from the globals: If __package__ (and __spec__.parent) are missing then import uses __name__, truncating the last segment if the module is a submodule rather than a package __init__.py (which it guesses from whether __path__ is defined). The __name__ attempt should fail if there is no parent package (top level modules), if __name__ is '__main__' (-m entry points), or both (scripts). That is, if both __name__ has no subcomponents and the module does not seem to be a package __init__ module then import should fail.

benjimin added 2 commits July 26, 2019 18:06

Regression tests for bpo-37409

effff9a

benjimin requested review from brettcannon, encukou, ericsnowcurrently, ncoghlan and warsaw as code owners July 26, 2019 08:29

the-knights-who-say-ni added the CLA signed label Jul 26, 2019

bedevere-bot added the awaiting review label Jul 26, 2019

brettcannon requested changes Aug 2, 2019

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Aug 2, 2019

benjimin and others added 7 commits August 5, 2019 23:19

Update Lib/test/test_builtin.py

201c40b

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

Update Lib/test/test_builtin.py

3fc5481

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

Update Lib/test/test_import/__init__.py

4a3f487

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

Clearer code

c240df5

Update Lib/test/test_builtin.py

5090880

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

📜🤖 Added by blurb_it.

57b47b7

Clarify test name

6f7f476

bedevere-bot added awaiting change review and removed awaiting changes labels Aug 7, 2019

brettcannon self-requested a review August 9, 2019 21:32

brettcannon requested changes Aug 9, 2019

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting change review labels Aug 9, 2019

Update Python/import.c

e0730ff

Co-Authored-By: Brett Cannon <54418+brettcannon@users.noreply.github.com>

bedevere-bot added awaiting change review and removed awaiting changes labels Aug 10, 2019

brettcannon self-requested a review August 20, 2019 18:52

brettcannon added the needs backport to 3.7 label Aug 20, 2019

ericsnowcurrently removed their request for review September 10, 2019 16:08

encukou approved these changes Sep 10, 2019

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Sep 10, 2019

brettcannon approved these changes Sep 11, 2019

View reviewed changes

brettcannon merged commit 92420b3 into python:master Sep 11, 2019

bedevere-bot removed the awaiting merge label Sep 11, 2019

miss-islington assigned brettcannon Sep 11, 2019

bedevere-bot removed the needs backport to 3.8 label Sep 11, 2019

brettcannon removed the needs backport to 3.7 label Sep 11, 2019

Uh oh!

bpo-37409: fix relative import with no parent #14956

bpo-37409: fix relative import with no parent #14956

Uh oh!

Conversation

benjimin commented Jul 26, 2019 • edited by brettcannon Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benjimin commented Jul 31, 2019

Uh oh!

brettcannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Aug 2, 2019

Uh oh!

benjimin commented Aug 7, 2019

Uh oh!

bedevere-bot commented Aug 7, 2019

Uh oh!

brettcannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Aug 9, 2019

Uh oh!

benjimin commented Aug 10, 2019

Uh oh!

bedevere-bot commented Aug 10, 2019

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington commented Sep 11, 2019

Uh oh!

bedevere-bot commented Sep 11, 2019

Uh oh!

miss-islington commented Sep 11, 2019

Uh oh!

miss-islington commented Sep 11, 2019

Uh oh!

bedevere-bot commented Sep 11, 2019

Uh oh!

brettcannon commented Sep 11, 2019

Uh oh!

brettcannon commented Sep 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

benjimin commented Jul 26, 2019 •

edited by brettcannon

Loading