"inspect" gets broken by some descriptors

[d-maurer]

BPO	1785
Nosy	@gvanrossum, @facundobatista, @d-maurer, @amauryfa, @pitrou, @merwok, @meadori, @ericsnowcurrently, @vpelletier
Files	inspectBug.py inspect-bug.patch inspect-and-pydoc-bug.patch inspect-and-pydoc-bug2.patch inspect-and-pydoc-bug3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-12-21.09:19:07.701>
created_at = <Date 2008-01-10.17:39:24.074>
labels = ['type-bug', 'library']
title = '"inspect" gets broken by some descriptors'
updated_at = <Date 2012-01-18.16:46:09.842>
user = 'https://github.com/d-maurer'

bugs.python.org fields:

activity = <Date 2012-01-18.16:46:09.842>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2011-12-21.09:19:07.701>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2008-01-10.17:39:24.074>
creator = 'dmaurer'
dependencies = []
files = ['9117', '9119', '9128', '23975', '23976']
hgrepos = []
issue_num = 1785
keywords = ['patch']
message_count = 20.0
messages = ['59675', '59676', '59677', '59684', '59712', '59743', '67777', '116863', '149242', '149606', '149607', '149968', '149970', '149973', '150005', '151544', '151545', '151548', '151549', '151550']
nosy_count = 13.0
nosy_names = ['gvanrossum', 'facundobatista', 'dmaurer', 'amaury.forgeotdarc', 'pitrou', 'flub', 'eric.araujo', 'meador.inge', 'jotr', 'BreamoreBoy', 'python-dev', 'eric.snow', 'vpelletier']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1785'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[d-maurer]

The inspect functions "getmembers(cls)" and "classify_class_attrs(cls)"
require that for a class *cls* each name in "dir(cls)" can be retrieved
by "getattr(cls, name)". While this holds for usual class attributes, it
may well fail for descriptors (descriptors set by 'zope.interface' are a
real world example for this). Attached it as small script that
demonstrates the problem.

The bug affects 'pydoc' and the built in 'help' (which is in fact
'pydoc.help'). While 'pydoc' and 'help' do not break completely, they
can not present meaningful information for classes with some descriptors

[gvanrossum]

Please submit a patch.

[d-maurer]

In "dm.zdoc" (a "pydoc" wrapper for Zope) I simply filter out all names
returned by "dir" which cannot be "getattr"ed.
But, I am not sure, that this is good enough to be accepted as a patch
(although it already improves upon the current state)

[pitrou]

This is my attempt at a patch for this. It fixes inspect.getmembers and
inspect.classify_class_attrs to work with Dieter's example. I hope it
doesn't mess anything else.

As for pydoc, things look a bit more complicated... The annoying thing
is that the logic to browse object contents is duplicated inside the
different renderers (pydoc.Repr subclasses). Also, I couldn't find any
unit tests for pydoc. Are there any?

[pitrou]

For the record the same problem also happens with toscawidgets.

>>> from toscawidgets.widgets.forms import validators
>>> help(validators)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site.py", line 351, in __call__
    return pydoc.help(*args, **kwds)
  File "/usr/lib/python2.5/pydoc.py", line 1646, in __call__
    self.help(request)
  File "/usr/lib/python2.5/pydoc.py", line 1690, in help
    else: doc(request, 'Help on %s:')
  File "/usr/lib/python2.5/pydoc.py", line 1481, in doc
    pager(title % desc + '\n\n' + text.document(object, name))
  File "/usr/lib/python2.5/pydoc.py", line 324, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "/usr/lib/python2.5/pydoc.py", line 1072, in docmodule
    contents.append(self.document(value, key, name))
  File "/usr/lib/python2.5/pydoc.py", line 325, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "/usr/lib/python2.5/pydoc.py", line 1173, in docclass
    classify_class_attrs(object))
  File "/usr/lib/python2.5/pydoc.py", line 179, in classify_class_attrs
    return map(fixup, inspect.classify_class_attrs(object))
  File "/usr/lib/python2.5/inspect.py", line 246, in classify_class_attrs
    obj = getattr(cls, name)
  File
"/usr/lib/python2.5/site-packages/FormEncode-0.7.1-py2.5.egg/formencode/declarative.py",
line 105, in __get__
    obj = type.singleton()
  File
"/usr/lib/python2.5/site-packages/FormEncode-0.7.1-py2.5.egg/formencode/declarative.py",
line 166, in singleton
    setattr(cls, name, cls(declarative_count=cls.declarative_count))
TypeError: __init__() got an unexpected keyword argument 'declarative_count'

[pitrou]

Here is a (hopefully complete) patch against both pydoc and inspect. It
fixes one more bug compared to the previous one (descriptors can also
have a special __getattr__ in addition to __get__, which gave problems
when trying to access __classobj__).

[facundobatista]

Really don't know why this was assigned to me...

[BreamoreBoy]

I can't apply the patch to any current SVN version.

[amauryfa]

The 'type' object now has the same issue: __abstractmethods__ appears in dir(type) but type.__abstractmethods__ fails with an AttributeError.
See bpo-13581

[pitrou]

Updated patch for 3.2.

[pitrou]

Updated patch that adds a test for classifying builtin types (checks that inspect.classify_class_attrs(type) does not throw as in bpo-13581).

[python-dev]

New changeset 902f694a7b0e by Antoine Pitrou in branch '3.2':
Issue bpo-1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/902f694a7b0e

New changeset b08bf8df8eec by Antoine Pitrou in branch 'default':
Issue bpo-1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/b08bf8df8eec

[python-dev]

New changeset 13f56cd8dec1 by Antoine Pitrou in branch '2.7':
Issue bpo-1785: Fix inspect and pydoc with misbehaving descriptors.
http://hg.python.org/cpython/rev/13f56cd8dec1

[pitrou]

Now fixed in all 3 branches.

[merwok]

Great stuff.

[vpelletier]

This change causes the following behaviour:

>>> import inspect
>>> class B(object):
...   def f(self):
...     pass
... 
>>> inspect.getmembers(B, inspect.ismethod)
[]

While I would expect the result to contain f:

>>> inspect.ismethod(B.f)
True

Isn't this a regression ?

Regards,
Vincent Pelletier

[vpelletier]

Sorry, I forgot to mention I'm using python2.7 .

[pitrou]

Thanks for noticing. The doc for ismethod() says:

“Return true if the object is a bound method written in Python.”

and the docstring agrees with that:

“Return true if the object is an instance method. [...]”

So the change isn't properly a regression when reading the docs. On the other hand, it's true that some code may rely on the previous behaviour, and the discrepancy between getmembers() and a manual test can be confusing.

By the way, Python 3 has ismethod() right:

>>> class B:
...   def f(self): pass
... 
>>> inspect.ismethod(B.f)
False
>>> inspect.ismethod(B().f)
True

[python-dev]

New changeset f824744557ba by Antoine Pitrou in branch '2.7':
Revert part of 13f56cd8dec1 (issue bpo-1785) to avoid breaking getmembers() with unbound methods.
http://hg.python.org/cpython/rev/f824744557ba

[pitrou]

I've backed out the part of the changeset that "fixed" getmembers(), so the old behaviour is restored. Other parts of the changeset (that e.g. fixed pydoc) have not been reverted.