Enhance inspect.getdoc to follow inheritance chains #59787
Comments
|
Currently, if you override a method from a base class, the docstring is not inherited, even if it remains accurate. This issue proposes an enhancement to inspect.getdoc() that allows the docstring to be retrieved from the inheritance hierarchy in the case where it is not overridden in the subclass by providing an explicit docstring. Specifically, in the case where obj.__doc__ is None, and either the first parameter is a bound method, or a class object is passed in as the second parameter, inspect.getdoc will search the MRO based on obj.__name__ until it finds an attribute with a non-None __doc__ value. (In Python 2, this could have been automatic for both bound and unbound methods. Unfortunately, there are no unbound methods in Python 3, so the second parameter is needed to handle the unbound method case) |
ncoghlan
added
stdlib
|
I have just the thing for this, but haven't had a chance to put a patch together. I might be able to get to it in the next week if Dave "bitey" Beazley isn't too much of a distraction. :) |
|
Hello! Here's a patch. Currently it lacks doc updates, but if the solution is okay, then I could provide them. |
|
Yury, Nick, how is my latest patch? |
|
@Claudiu, you should also add this test and make sure that it passes: class Parent:
__doc__ = 'some documentation'
class Child(Parent):
__doc__ = Noneassert getdoc(Child) is None In other words -- we use __doc__ defined in parent classes only when there was no __doc__ in children's __dict__s. |
|
Alright, I'll update my patch later this week. |
|
Yury, regarding your last message, is it actually possible to have a subclass which doesn't have a __doc__ attribute in its __dict__, except using slots? __doc__ seems to be set to None every time if it's not specified, so I don't know how could I detect the case where the client sets '__doc__ = None' himself. The following example might be more explanatory: >>> class A:
... __doc__ = "a"
...
>>> inspect.getdoc(A)
'a'
>>> inspect.getdoc(A())
'a'
>>> class B(A):
... __doc__ = None
...
>>> vars(B)
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>> B.__dict__
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>> class C(A): pass
...
>>> vars(C)
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>>Nevertheless, my patch ignores this case, since it operates only on methods. When trying to do inspect.getdoc(Child, parent=Parent), it will try to look for an attribute 'Child' in the mro of Parent and thus it will return None, since this doesn't exist (this can actually be a problem, if that attribute actually exist). |
|
The patch adds support only for unbound methods and requires additional parameter for this. It is not what should be done in this issue at all. I'm interested in this issue so I'll write a patch. It should be easier to write it myself than describe it. |
Just curios, which part works only for unbound methods?
Well, some hints for the right way to do this would have been appreciated. |
|
Here is a patch that supports classes, bound and unbound methods, class methods, properties, method and class descriptors. |
|
Serhiy's patch looks good to me. I'd completely missed that __qualname__ could be used to avoid needing a second argument even when handling objects other than bound methods. That's very cool, and I can see why you figured it was easier to just write the patch than explain what you had in mind. |
|
New changeset 157b7055bb9d by Serhiy Storchaka in branch 'default': |
|
Unfortunately this enhancement breaks test_enum. ====================================================================== Traceback (most recent call last):
File "/home/serhiy/py/cpython/Lib/test/test_enum.py", line 1609, in test_pydoc
self.assertEqual(result, expected_text)
AssertionError: 'Help[68 chars]n | Generic enumeration.\n | \n | Derive fr[876 chars]ing.' != 'Help[68 chars]n | Method resolution order:\n | Color\n[782 chars]ing.'
Help on class Color in module test.test_enum:
class Color(enum.Enum)
- | Generic enumeration.
- |
- | Derive from this class to define new enumerations.
- |
| Method resolution order:
| Color
| enum.Enum
| builtins.object
|
| Data and other attributes defined here:
|
| blue = <Color.blue: 3>
|
| green = <Color.green: 2>
|
| red = <Color.red: 1>
|
| | Data descriptors inherited from enum.Enum: ---------------------------------------------------------------------- |
|
New changeset 47a61a1c97b3 by Serhiy Storchaka in branch 'default': |
|
Changed test_enum to make buildbots green, but perhaps the docstring of Enum should be changed, because it now is used for all Enum subclasses that doesn't define a docstring explicitly. An alternative solution is to set __doc__ of Enum subclasses to an empty string if the docstring is not defined explicitly. |
|
I filed issue bpo-23900 to consider the question of the default docstring for Enum subclasses. |
|
Sometimes the doc string for the overridden method does not make much sense in the context of the subclass. Just wondering if this was considered; it seems like a fairly serious downside to this new feature. E.g. in a package I am reviewing, there is a class that inherits HTMLParser and converts HTML to PDF. There is no doc string, so previously there was just the signature in the “pydoc” output. Now the output looks like: | __init__(self, pdf, image_map=None) The second paragraph mentions parameters and settings of the internal base class, which doesn’t make much sense for the subclass. |
|
Sounds like good incentive to add good docstrings. :) |
|
I were aware that this can propagate some not well appropriate docstrings from abstract base classes, but Martin expose worse problem: inheriting a docstring by the method with changed signature. Perhaps we should check if a signature of overriding method is compatible with a signature of base method. But this is hard to implement, and impossible in some cases until all builtins will converted to Argument Clinic. |
|
I know it will take some time to add doc strings to places where they are missing, but I would not roll-back the patch for that. File new issues and get the over-riding methods properly documented. |
|
My example was taken from a package that tends to “properly document” everything in separate documentation files, rather than in the source code. I don’t really think Python should dictate if and how one document their code based on what base classes they use, and pydoc is still useful with code that has no doc strings. But if this is to be the way of the future, perhaps a warning in the “What’s New” page might be a good idea. The inspect.getdoc() documentation should probably also have a “Changed in version 3.5” warning. I guess it would be less automatic, but maybe another option might have been to add an @inherit_docstring decorator, or some explicit way to say a particular API (re)implements an (abstract) API documented elsewhere. |
|
Yes, this was a deliberate change to "flip the default" as to which subclasses get bad docstring behaviour. In the status quo, if you provide a subclass method which does basically the same thing as the parent class, you lose the docstring unless you duplicate it, meaning you have to choose between bad docstrings and a maintainability problem due to content duplication. With this change, you only need a custom docstring in the subclass if you've changed the method behaviour enough that the parent class docstring no longer makes any sense. If you just want to suppress the docstring entirely, then you'll need to specify an empty docstring. This is potentially worth a note in the "Porting to Python 3.5" section of the What's New document, but it's an intended consequence of the change. |
|
getdoc-news.patch suggests some wording to add to What’s New, and also adds a “Changed in version 3.5” note to inspect.getdoc(). BTW I also noticed that the class doc strings are not inherited from object.__doc__, although method doc strings _are_ inherited from object(), such as object.__init__.__doc__. The current documentation suggests that the class doc string “The most base type” should also be inherited. $ cat module.py
class SomeClass:
'''CLASS DOCSTRING'''
def __init__(self):
'''METHOD DOCSTRING'''
$ ./python -m pydoc module.SomeClass # Doc strings intact
[. . .]
module.SomeClass = class SomeClass(builtins.object)
| CLASS DOCSTRING
|
| Methods defined here:
|
| __init__(self)
| METHOD DOCSTRING
| [. . .]
$ ./python -OOm pydoc module.SomeClass # Method inherited, class stripped
[. . .]
module.SomeClass = class SomeClass(builtins.object)
| Methods defined here:
|
| __init__(self)
| Initialize self. See help(type(self)) for accurate signature.
| [. . .]I also wonder how well this feature would work when someone tries to override a base method by using a mix-in type class. |
|
Merged the current What’s New page in getdoc-news.v2.patch. Is there any interest in applying this? |
|
New changeset 4476b578b8fd by Berker Peksag in branch '3.5': New changeset e0f4a5f09bfa by Berker Peksag in branch 'default': |
|
Applied, thanks! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: