Importlib: Loader issue

[afourmy]

When loading a module dynamically with the following code:

from importlib.util import spec_from_file_location, module_from_spec
from pathlib import Path

file = Path.cwd() / etc...
spec = spec_from_file_location(str(file), str(file))
spec.loader.exec_module(module_from_spec(spec))

Mypy fails with the following errors:

test_loader.py:7: error: Item "_Loader" of "Optional[_Loader]" has no attribute "exec_module"
test_loader.py:7: error: Item "None" of "Optional[_Loader]" has no attribute "exec_module"

For the first error, from what I understand, the problem is that _Loader does not have exec_module while Loader does, and ModuleSpec.loader is set to Optional[_Loader].
Removing the underscore here

typeshed/stdlib/3/_importlib_modulespec.pyi

Line 22 in ac8f5da

loader = ... # type: Optional[_Loader]

seems to work, but I'm not sure this is the right way to fix it.
The second error is due to Optional... it appears that None is a valid loader value (when the loader is not found).

[srittau]

PEP 302 defines a loader like this:

A finder object has a single method:

finder.find_module(fullname, path=None)

The glossary does this as well. That's the reason there is a loader protocol in the importlib stub. I think the best bet is to assert isinstance(Loader, spec.loader) in the code above.

[afourmy]

Do you mean importlib.abc.Loader ?

With the following code:

from importlib.abc import Loader
from importlib.util import spec_from_file_location
from pathlib import Path

file = Path.cwd() / etc...
spec = spec_from_file_location(str(file), str(file))
assert isinstance(Loader, spec.loader)

I get the following exception: TypeError: isinstance() arg 2 must be a type or tuple of types (no idea why, apparently spec.loader is an instance of _frozen_importlib_external.SourceFileLoader)

Anyway, I don't think an assert would fix the first error, i.e _Loader not having an exec_module method ?

[srittau]

Sorry, I got the order of isinstance() arguments mixed up. It should be isinstance(spec.loader, Loader). This will get rid of both errors, since mypy now knows that spec.loader is in reality a Loader instance, which has the exec_module() method.

[afourmy]

Indeed, it works, thanks !

[Bowrna]

@srittau I am facing a similar issue too. I have added the assert statement. But still it throws error. Following is my code snippet:

from importlib.abc import Loader
spec = importlib.util.find_spec(package_name)
 if spec is not None:
    module = importlib.util.module_from_spec(spec)
    sys.modules[package_name] = module
    assert isinstance(spec.loader, Loader)
    spec.loader.exec_module(module)
     console.print(f"\n[blue]Package name {package_name} is installed and imported to run static check test")
else:
      console.print(f"\n[red]Error: Package name {package_name} is not installed. Please install using https://pre-commit.com/#install to continue[/]\n")

Below is the error I am getting

dev/breeze/src/airflow_breeze/breeze.py:290: error: Item "_LoaderProtocol" of
"Optional[_LoaderProtocol]" has no attribute "exec_module"  [union-attr]
            spec.loader.exec_module(module)
            ^
dev/breeze/src/airflow_breeze/breeze.py:290: error: Item "None" of
"Optional[_LoaderProtocol]" has no attribute "exec_module"  [union-attr]
            spec.loader.exec_module(module)
            ^

[srittau]

@Bowrna I just tried this self-contained example with mypy 0.931 and it passed without problems:

import importlib.util
from importlib.abc import Loader

spec = importlib.util.find_spec("")
if spec is not None:
    module = importlib.util.module_from_spec(spec)
    assert isinstance(spec.loader, Loader)
    spec.loader.exec_module(module)

If you still have problems, could you ask in our forums? https://github.com/python/typing/discussions