Use explicit imports to avoid future AttributeErrors

[tirkarthi]

The CPython internals could change in future and using explicit imports will help in safeguarding.

Similar issue in gcloud : https://issuetracker.google.com/issues/170125513
Found in yozik04/vallox_websocket_api#15

[tincho9]

Is it be possible to merge this in order to support python 3.9? It's currently breaking python 3.9 support in libraries that use construct.

[slovdahl]

Is there anything we can do to help get this moved forward?

This one sounds a bit similar too: https://bugs.python.org/issue31318

[slovdahl]

Interestingly enough, with the minimal reproducer below I can reproduce the same type of error on my Macbook (where I originally got it) and on a Raspberry Pi 4 running Raspbian Buster, but not on another Raspberry Pi 4 running Ubuntu 20.04. On my Macbook, Python is coming from https://github.com/pyenv/pyenv, in Raspbian and Ubuntu 20.04 Python is installed with apt-get.

macOS:

$ python3
Python 3.10.2 (main, Feb  5 2022, 17:43:08) [Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> importlib.util.find_spec("re")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'
>>> import importlib.util
>>> importlib.util.find_spec("re")
ModuleSpec(name='re', loader=<_frozen_importlib_external.SourceFileLoader object at 0x1071adfc0>, origin='/Users/slovdahl/.pyenv/versions/3.10.2/lib/python3.10/re.py')

Raspbian Buster:

$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> importlib.util.find_spec("re")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'
>>> import importlib.util
>>> importlib.util.find_spec("re")
ModuleSpec(name='re', loader=<_frozen_importlib_external.SourceFileLoader object at 0xb6709fd0>, origin='/usr/lib/python3.7/re.py')

Ubuntu 20.04:

$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> importlib.util.find_spec("re")
ModuleSpec(name='re', loader=<_frozen_importlib_external.SourceFileLoader object at 0xffffb91a63a0>, origin='/usr/lib/python3.8/re.py')

[slovdahl]

@arekbulski what are your thoughts about this?

[arekbulski]

Library is tested against python 3.9 since a log time ago. Since 2 days ago it is also tested against 3.10. It works without a hitch. So, you will convince me please that the changes in question are necessary.

[slovdahl]

I would love to have some kind of reproducer to provide, but I have not been able to pinpoint why exactly it fails for me (and others), and why it for me only fails on macOS. Could it have something to do with how Python is built in a specific situation? Or something in the surrounding environment that somehow impacts it.

According to a comment by a Python committer in https://bugs.python.org/issue31318, because importlib.util is a submodule of importlib, it should need its own import. But it's certainly interesting that it seems to work without it in most cases.

Do you have any ideas for how to troubleshoot it further?

[arekbulski]

Do you have any ideas for how to troubleshoot it further?

Unfortunately no. I can add the imports "just in case" and we see if that changes anything maybe?

[slovdahl]

That would be appreciated 👍 I'm pretty sure it should fix it for me at least. I tried doing that same change in my local site-packages, and it worked afterwards.

[tirkarthi]

Closing since the changes are merged in 83f9080