Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importlib.find_spec raises AttributeError when parent is not a package/module #74621

Closed
tkhyn mannequin opened this issue May 23, 2017 · 8 comments
Closed

importlib.find_spec raises AttributeError when parent is not a package/module #74621

tkhyn mannequin opened this issue May 23, 2017 · 8 comments
Labels
3.7 stdlib type-bug

Comments

@tkhyn
Copy link
Mannequin

@tkhyn tkhyn mannequin commented May 23, 2017

BPO 30436
Nosy @brettcannon, @ncoghlan, @ned-deily, @ericsnowcurrently, @zvyn, @tkhyn, @miss-islington
PRs
  • #1899
  • #7385
  • #7469
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-06-14.21:35:16.081>
    created_at = <Date 2017-05-23.00:47:04.511>
    labels = ['3.7', 'type-bug', 'library']
    title = 'importlib.find_spec raises AttributeError when parent is not a package/module'
    updated_at = <Date 2018-06-07.06:21:45.463>
    user = 'https://github.com/tkhyn'

    bugs.python.org fields:

    activity = <Date 2018-06-07.06:21:45.463>
    actor = 'miss-islington'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-06-14.21:35:16.081>
    closer = 'brett.cannon'
    components = ['Library (Lib)']
    creation = <Date 2017-05-23.00:47:04.511>
    creator = 'tkhyn'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 30436
    keywords = []
    message_count = 8.0
    messages = ['294209', '294280', '294281', '294383', '294903', '296041', '318895', '318898']
    nosy_count = 7.0
    nosy_names = ['brett.cannon', 'ncoghlan', 'ned.deily', 'eric.snow', 'zvyn', 'tkhyn', 'miss-islington']
    pr_nums = ['1899', '7385', '7469']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue30436'
    versions = ['Python 3.7']

    @tkhyn
    Copy link
    Mannequin Author

    @tkhyn tkhyn mannequin commented May 23, 2017

    Hello, I stumbled upon this issue when using the module_has_submodule function in Django, which raised an exception when trying to import a dotted path such as parent.module when parent does not exist or is not a package. I would expect (as well as the django devs, apparently) find_spec to return None instead of raising an AttributeError or ModuleNotFoundError.

    Unless you think Django or any package making use of importlib.find_spec should handle these exceptions, the fix is quite simple.

    Steps to reproduce (with Python 3.6.1):

    touch parent.py
    python3.6
    >>> from importlib.util import find_spec
    >>> find_spec('parent.module')
      File "C:\Python\3.6\Lib\importlib\util.py", line 89, in find_spec
        return _find_spec(fullname, parent.__path__)
    AttributeError: module 'parent' has no attribute '__path__'
    >>> find_spec('invalid_parent.module')
      File "C:\Python\3.6\Lib\importlib\util.py", line 88, in find_spec
        parent = __import__(parent_name, fromlist=['__path__'])
    ModuleNotFoundError: No module named 'invalid_parent'

    The fix is quite simple, replacing

        if fullname not in sys.modules:
            parent_name = fullname.rpartition('.')[0]
            if parent_name:
                # Use builtins.__import__() in case someone replaced it.
    
                parent = __import__(parent_name, fromlist=['__path__'])
                return _find_spec(fullname, parent.__path__)
        else:
    
                return _find_spec(fullname, None)
    by:
    
        if fullname not in sys.modules:
            parent_name = fullname.rpartition('.')[0]
            if parent_name:
                # Use builtins.__import__() in case someone replaced it.
                try:
                    parent = __import__(parent_name, fromlist=['__path__']).__path__
    
                except (AttributeError, ModuleNotFoundError):
                    # parent is not a package
                    return None
            else:
                parent = None
            return _find_spec(fullname, parent)

    in importlib.util.find_spec.

    @tkhyn tkhyn mannequin added the stdlib label May 23, 2017
    @brettcannon
    Copy link
    Member

    @brettcannon brettcannon commented May 23, 2017

    The key thing to think about is do you think find_spec("parent.module") is working with a single thing called "parent.module" or is it working with two separate things of "parent" and "module" which happens to be contained on "parent"? If you take the former view then you get the current semantics, but if you view it as the latter then you get the semantics you're suggesting, tkhyn.

    My inclination is for the former semantics (i.e. think of it as a really long name for a specific module where it turns out the name is broken). If you look at it as find_spec(".submodule", package="parent") this also visually supports the idea that parent modules shouldn't trigger a None return. Finally, this would break any code that expects the current semantics.

    So thanks for the bug report, but I'm going to close this as "not a bug".

    @brettcannon brettcannon added invalid type-bug labels May 23, 2017
    @tkhyn
    Copy link
    Mannequin Author

    @tkhyn tkhyn mannequin commented May 23, 2017

    Ok, thanks for the reply. Actually the thing that bothered me was the AttributeError exception. I would probably not have opened a ticket should find_spec have raised a ModuleNotFoundError (in line with import_module).

    Would you consider catching the AttributeError (which means detecting if parent_name relates to a package) to raise a ModuleNotFoundError instead more appropriate?

    @brettcannon
    Copy link
    Member

    @brettcannon brettcannon commented May 24, 2017

    Here is why Python does when importing a module that lacks __path__:

    >>> import importlib
    >>> del importlib.__path__
    >>> import importlib.util
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'importlib.util'; 'importlib' is not a package

    So yes, we should change find_spec() to raise ModuleNotFoundError to match (although only in Python 3.7 since this is a breaking change).

    @brettcannon brettcannon reopened this May 24, 2017
    @brettcannon brettcannon removed the invalid label May 24, 2017
    @brettcannon brettcannon changed the title importlib.find_spec raises AttributeError/ModuleNotFoundError when parent is not a package/module importlib.find_spec raises AttributeError when parent is not a package/module May 24, 2017
    @zvyn
    Copy link
    Mannequin

    @zvyn zvyn mannequin commented Jun 1, 2017

    I added a PR changing the exception raised as suggested, reviews welcome!

    @brettcannon
    Copy link
    Member

    @brettcannon brettcannon commented Jun 14, 2017

    New changeset 8c3f05e by Brett Cannon (Milan Oberkirch) in branch 'master':
    bpo-30436: Raise ModuleNotFoundError for importlib.util.find_spec() when parent isn't a package (GH-1899)
    8c3f05e

    @ned-deily
    Copy link
    Member

    @ned-deily ned-deily commented Jun 7, 2018

    New changeset fffeb6f by Ned Deily (Zackery Spytz) in branch 'master':
    bpo-30436: Add missing space in importlib.util.find_spec() error message (GH-7385)
    fffeb6f

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 7, 2018

    New changeset d2a2af0 by Miss Islington (bot) in branch '3.7':
    bpo-30436: Add missing space in importlib.util.find_spec() error message (GH-7385)
    d2a2af0

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 stdlib type-bug
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants