incomplete signature with help function using typing

[DavidEFrancoG]

BPO	27989
Nosy	@gvanrossum, @1st1, @ilevkivskyi, @TeamSpen210
Files	typing-pydoc.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-10-22.14:58:34.773>
created_at = <Date 2016-09-07.02:19:50.900>
labels = ['type-bug', 'library']
title = 'incomplete signature with help function using typing'
updated_at = <Date 2016-10-22.15:01:01.025>
user = 'https://bugs.python.org/DavidEFrancoG'

bugs.python.org fields:

activity = <Date 2016-10-22.15:01:01.025>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2016-10-22.14:58:34.773>
closer = 'gvanrossum'
components = ['Library (Lib)']
creation = <Date 2016-09-07.02:19:50.900>
creator = 'David E. Franco G.'
dependencies = []
files = ['45188']
hgrepos = []
issue_num = 27989
keywords = ['patch']
message_count = 15.0
messages = ['274702', '274734', '274787', '274814', '274879', '274881', '274889', '274904', '279190', '279191', '279192', '279193', '279194', '279195', '279197']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'python-dev', 'yselivanov', 'levkivskyi', 'Spencer Brown', 'David E. Franco G.']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27989'
versions = ['Python 3.5']

[DavidEFrancoG]

the issue is that when calling help on a function annotated with typing, all the relevant information is lost, for example

from typing import List, Any, Iterator, Tuple

def foo(data:List[Any]) -> Iterator[ Tuple[int,Any] ]:
    ...
 
when calling help on that

>>> help(foo)
Help on function foo in module __main__:

foo(data:typing.List) -> typing.Iterator

>>

all the information is lost, the output should look like this

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[ Tuple[int, Any] ]:

>>

where all the information that I put in the annotation is preserved and the typing.* are eliminated since they only add unnecessary noise

while reporting this issue in the typing module (https://github.com/python/typing/issues/279) I was told that is a bug with the inspect module and that help may need modification.

Thank for your time.

[gvanrossum]

It seems the output produced here is generated by inspect.signature(), which is called by pydoc in this case (in both versions of docroutine()).

I don't know if the right thing to do is to change inspect.signature() here, or to change pydoc to use something else to format the argument list.

[TeamSpen210]

More precisely, the issue is with inspect.formatannotation(), which overrides/ignores the repr if the annotation is an instance of type. Perhaps that should be changed to also check that __repr__ is type's repr.

[DavidEFrancoG]

as that is the case, how about this as a solution:

def formatannotation(annotation, base_module=None):
    if isinstance(annotation, type):
        if annotation.__module__ in ('builtins', base_module):
            return annotation.__qualname__
        elif annotation.__module__ in ('typing', base_module):       
            return repr(annotation).replace("typing.","")
        return annotation.__module__+'.'+annotation.__qualname__
    return repr(annotation)

the same way that it check for builtins, do it for typing and clean up a little.

With that change the result with the example is

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[Tuple[int, Any]]

>>

[gvanrossum]

That sounds a fine solution (except the elif should just test for in 'typing'). Can one of you prepare a patch? I think it should be fine
to fix this in 3.5 as well.

There should be a unit test for this.

[TeamSpen210]

It might be better to just change the if statement to 'if isinstance(annotation, type) and type(annotation).__repr__ is type.__repr__:'. That would make it fallback for any metaclass which overrides repr, instead of special-casing typing. That also ensures 'typing.' is still in the name, since these aren't builtins.

[gvanrossum]

I've lost you -- why don't you upload a patch?

[DavidEFrancoG]

I think that removing the "typing." is for the best, with the example above

>>> help(foo)
Help on function foo in module __main__:

foo(data:typing.List[typing.Any]) -> typing.Iterator[typing.Tuple[int, typing.Any]]

>>

leaving the "typing." produce result that I think are ugly and distracting, not only that, is unnecessary long to convey the same information that can be in a more neat way without it, and more so while more complicated/long the signature is.

just compare the above with this

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[Tuple[int, Any]]:

>>

which is a clear winner to me.

Or perhaps alongside modifying inspect.formatannotation also change the repr in typing to exclude the typing. or like with for instance TypeVar produce a repr that include some marker instead, like ~ and in that way indicate that one is using typing object resulting in something like this

>>> help(foo)
Help on function foo in module __main__:

foo(data:~List[Any]) -> ~Iterator[~Tuple[int, ~Any]]:

>>

which is a little weird but still neat

[ilevkivskyi]

Here is the patch according to the discussion (modifying inspect).

I didn't change the rendering of docs for classes (neither stripped 'typing.' nor changed __bases__ to __orig_bases__). First, collections.abc.X are widely used as base classes, so that plain Mapping could be confused with collections.abc.Mapping. Second, seeing the actual runtime type-erased bases suggests that one should use isinstance() and issubclass() with those (not with, e.g., Mapping[int, str], the latter will raise TypeError).

[gvanrossum]

Hm, I actually like the original proposal better. Perhaps collections.abc.Mapping is more common than typing.Mapping, but is it more common *in function annotations*? I don't think so.

Also, I like showing e.g. Iterator[Tuple[int, Any]] rather than just Iterator. This is documentation we're talking about, and the parameter types are very useful as documentation. (However, abbreviating List[Any] as List is fine, since they mean the same thing.)

[ilevkivskyi]

For function annotations I did as originally proposed. In my previous comment I was talking about documentation for classes. For example:

class C(Generic[T], Mapping[int, str]): ...
pydoc.render_doc(C)

will show "class C(typing.Mapping)".

while for function annotations typing is indeed much more common so that pydoc.render_doc(foo) will show

foo(data: List[Any]) -> Iterator[Tuple[int, Any]]

[gvanrossum]

OK, sounds good then. I guess most of the work was in typing.py, not in inspect. :-)

[ilevkivskyi]

Actually, for classes, it is probably worth adding a separate section "Generic type info" that will render information using __orig_bases__, __parameters__, and __args__. At the same time the "header" will be the same as now, listing runtime __bases__.

What do you think about this? Should I open a separate issue?

[python-dev]

New changeset dc030d15f80d by Guido van Rossum in branch '3.5':
Issue bpo-27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L.
https://hg.python.org/cpython/rev/dc030d15f80d

New changeset 3937502c149d by Guido van Rossum in branch '3.6':
Issue bpo-27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L. (3.5->3.6)
https://hg.python.org/cpython/rev/3937502c149d

New changeset 62127e60e7b0 by Guido van Rossum in branch 'default':
Issue bpo-27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L. (3.6->3.7)
https://hg.python.org/cpython/rev/62127e60e7b0

[gvanrossum]

Honestly I think pydoc is already too verbose. It would be better if the class header looked more like what was written in the source code -- that is the most compact way to render it. I say open a separate issue since this issue is about functions.