pydoc help("modules keyword") is failing when a module throws an exception at the import time

[ChristianSPerone]

BPO	12894
Nosy	@ned-deily
Superseder	bpo-7425: Improve the robustness of "pydoc -k" in the face of broken modules

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-09-05.01:31:25.607>
created_at = <Date 2011-09-05.01:15:57.381>
labels = ['type-bug', 'library']
title = 'pydoc help("modules keyword") is failing when a module throws an exception at the import time'
updated_at = <Date 2011-09-05.01:31:25.592>
user = 'https://bugs.python.org/ChristianSPerone'

bugs.python.org fields:

activity = <Date 2011-09-05.01:31:25.592>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2011-09-05.01:31:25.607>
closer = 'ned.deily'
components = ['Library (Lib)']
creation = <Date 2011-09-05.01:15:57.381>
creator = 'Christian.S..Perone'
dependencies = []
files = []
hgrepos = []
issue_num = 12894
keywords = []
message_count = 2.0
messages = ['143502', '143504']
nosy_count = 2.0
nosy_names = ['ned.deily', 'Christian.S..Perone']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '7425'
type = 'behavior'
url = 'https://bugs.python.org/issue12894'
versions = ['Python 2.6', 'Python 2.7']

[ChristianSPerone]

Pydoc is failing when running the ModuleScanner().run() when you execute:

help("modules keyword")

... if some module throws an exception at the import time.
See this example:

>> help("modules convolve")

Here is a list of matching modules. Enter any module name to get more help.

numpy.numarray.convolve
scipy.fftpack.convolve
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site.py", line 467, in __call__
    return pydoc.help(*args, **kwds)
  File "C:\Python27\lib\pydoc.py", line 1727, in __call__
    self.help(request)
  File "C:\Python27\lib\pydoc.py", line 1768, in help
    self.listmodules(split(request)[1])
  File "C:\Python27\lib\pydoc.py", line 1873, in listmodules
    apropos(key)
  File "C:\Python27\lib\pydoc.py", line 1975, in apropos
    ModuleScanner().run(callback, key)
  File "C:\Python27\lib\pydoc.py", line 1938, in run
    for importer, modname, ispkg in pkgutil.walk_packages(onerror=onerror):
  File "C:\Python27\lib\pkgutil.py", line 125, in walk_packages
    for item in walk_packages(path, name+'.', onerror):
  File "C:\Python27\lib\pkgutil.py", line 125, in walk_packages
    for item in walk_packages(path, name+'.', onerror):
  File "C:\Python27\lib\pkgutil.py", line 125, in walk_packages
    for item in walk_packages(path, name+'.', onerror):
  File "C:\Python27\lib\pkgutil.py", line 125, in walk_packages
    for item in walk_packages(path, name+'.', onerror):
  File "C:\Python27\lib\pkgutil.py", line 110, in walk_packages
    __import__(name)
  File "C:\Python27\lib\site-packages\wx-2.8-msw-unicode\wx\lib\pubsub\core\arg1\__init__.py", line 16, in <module>
    raise RuntimeError(msg)
RuntimeError: Should not import this directly, used by pubsub.core if applicable

After suppressing the exception by changing the "apropos()" method inside the pydoc.py:

Line:
ModuleScanner().run(callback, key)

To:
ModuleScanner().run(callback, key, onerror=lambda error: error)

The problem is fixed, but I really don't know if this is the best way to suppress this and if we should suppress these exceptions, I think that as the intent of the help("modules keyword") is to find things with the keyword in the name, we shouldn't let the exception be propagated to the user.

[ned-deily]

Thanks for the suggested fix. It looks like the problem and fix have already been documented in bpo-7425. Merging this into the existing issue.