Trying to complete import six.moves causes IPython crash

[cielavenir]

• six 1.16.0
• IPython 5.10.0
• Python 3.9.2 (/ 2.7.18)

In [2]: import six.moves.a[TAB]                                                                                                                                                                                      
Traceback (most recent call last):
  File "/usr/lib/python3.9/dbm/gnu.py", line 4, in <module>
    from _gdbm import *
ModuleNotFoundError: No module named '_gdbm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/bin/ipython3", line 5, in <module>
    start_ipython()
  File "/opt/lib/python3.9/site-packages/IPython/__init__.py", line 119, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/opt/lib/python3.9/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/opt/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 355, in start
    self.shell.mainloop()
  File "/opt/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 495, in mainloop
    self.interact()
  File "/opt/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 478, in interact
    code = self.prompt_for_code()
  File "/opt/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 371, in prompt_for_code
    document = self.pt_cli.run(
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/interface.py", line 394, in run
    self.eventloop.run(self.input, self.create_eventloop_callbacks())
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/eventloop/posix.py", line 164, in run
    t()
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/eventloop/posix.py", line 83, in read_from_stdin
    inputstream.feed(data)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/terminal/vt100_input.py", line 395, in feed
    self._input_parser.send(c)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/terminal/vt100_input.py", line 304, in _input_parser_generator
    self._call_handler(match, prefix)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/terminal/vt100_input.py", line 337, in _call_handler
    self.feed_key_callback(KeyPress(key, insert_text))
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/interface.py", line 1023, in feed_key
    cli.input_processor.process_keys()
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/key_binding/input_processor.py", line 201, in process_keys
    self._process_coroutine.send(key_press)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/key_binding/input_processor.py", line 158, in _process
    self._call_handler(matches[-1], key_sequence=buffer)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/key_binding/input_processor.py", line 228, in _call_handler
    handler.call(event)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/key_binding/registry.py", line 31, in call
    return self.handler(event)
  File "/opt/lib/python3.9/site-packages/prompt_toolkit/key_binding/bindings/completion.py", line 50, in display_completions_like_readline
    completions = list(b.completer.get_completions(b.document, complete_event))
  File "/opt/lib/python3.9/site-packages/IPython/terminal/ptutils.py", line 49, in get_completions
    used, matches = self.ipy_completer.complete(
  File "/opt/lib/python3.9/site-packages/IPython/core/completer.py", line 1169, in complete
    custom_res = self.dispatch_custom_completer(text)
  File "/opt/lib/python3.9/site-packages/IPython/core/completer.py", line 1085, in dispatch_custom_completer
    res = c(event)
  File "/opt/lib/python3.9/site-packages/IPython/core/completerlib.py", line 259, in module_completer
    return module_completion(event.line)
  File "/opt/lib/python3.9/site-packages/IPython/core/completerlib.py", line 236, in module_completion
    completion_list = try_import('.'.join(mod[:-1]), True)
  File "/opt/lib/python3.9/site-packages/IPython/core/completerlib.py", line 175, in try_import
    completions.extend( [attr for attr in dir(m) if
  File "/opt/lib/python3.9/site-packages/IPython/core/completerlib.py", line 176, in <listcomp>
    is_importable(m, attr, only_modules)])
  File "/opt/lib/python3.9/site-packages/IPython/core/completerlib.py", line 157, in is_importable
    return inspect.ismodule(getattr(module, attr))
  File "/opt/lib/python3.9/site-packages/six.py", line 97, in __get__
    result = self._resolve()
  File "/opt/lib/python3.9/site-packages/six.py", line 120, in _resolve
    return _import_module(self.mod)
  File "/opt/lib/python3.9/site-packages/six.py", line 87, in _import_module
    __import__(name)
  File "/usr/lib/python3.9/dbm/gnu.py", line 6, in <module>
    raise ImportError(str(msg) + ', please install the python3-gdbm package')
ImportError: No module named '_gdbm', please install the python3-gdbm package

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

gdbm (and tk) are optional packages but dir(six.moves) shows it.

Indeed, getattr(six.moves, 'dbm_gnu') raises ImportError.

However, it should not cause crashing IPython.

[krassowski]

IPython 5.10.0

Can you reproduce this using a modern version of IPython? I suspect it is unlikely that a patch would be backported so far back, the current version is 8.24 and there has been a complete refactor of the completer in 8.x series

[cielavenir]

• Python 3.9.19
• IPython 8.18.1

docker run --rm -it --entrypoint=bash python:3.9

rm /usr/local/lib/python3.*/lib-dynload/_gdbm.cpython*.so
python -m pip install six
python3 -m pip install IPython~=8.0
ipython

shows this, though IPython process does not crash

In [1]: import six.moves.a[TAB]

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completer.py", line 3171, in _complete
    result = matcher(context)
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completer.py", line 2707, in custom_completer_matcher
    matches = self.dispatch_custom_completer(context.token) or []
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completer.py", line 2747, in dispatch_custom_completer
    res = c(event)
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completerlib.py", line 276, in module_completer
    return module_completion(event.line)
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completerlib.py", line 253, in module_completion
    completion_list = try_import('.'.join(mod[:-1]), True)
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completerlib.py", line 186, in try_import
    completions.extend( [attr for attr in dir(m) if
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completerlib.py", line 187, in <listcomp>
    is_importable(m, attr, only_modules)])
  File "/usr/local/lib/python3.9/site-packages/IPython/core/completerlib.py", line 156, in is_importable
    return inspect.ismodule(getattr(module, attr))
  File "/usr/local/lib/python3.9/site-packages/six.py", line 97, in __get__
    result = self._resolve()
  File "/usr/local/lib/python3.9/site-packages/six.py", line 120, in _resolve
    return _import_module(self.mod)
  File "/usr/local/lib/python3.9/site-packages/six.py", line 87, in _import_module
    __import__(name)
  File "/usr/local/lib/python3.9/dbm/gnu.py", line 3, in <module>
    from _gdbm import *
ModuleNotFoundError: No module named '_gdbm'

If you suspect this is an IPython 8.18.1 bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

[krassowski]

Adding a catch for ModuleNotFoundError around inspect.ismodule call sounds reasonable to me, but then if reproducer involves deleting shared library from file disk I am not sure if this is really a bug or just a problem with the Python installation?

[cielavenir]

@krassowski gdbm (and tk) are optional packages. Not directly reproducible in python3 docker images but easily reproducible with ubuntu/debian not installing python3-gdbm. (Especially gdbm being GPL)

[cielavenir]

notifying @felixvd and @hemangandhi as our system does not install python-gdbm