inspect.getsource does not work with decorated functions

[micheles]

BPO	1764286
Nosy	@abalkin, @pitrou, @tiran, @PCManticore, @1st1, @eric-wieser
Files	inspect.py.diff inspect_getsource.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2014-09-26.21:36:35.226>
created_at = <Date 2007-07-31.10:16:40.000>
labels = ['type-bug', 'library']
title = 'inspect.getsource does not work with decorated functions'
updated_at = <Date 2018-08-02.06:34:15.069>
user = 'https://bugs.python.org/micheles'

bugs.python.org fields:

activity = <Date 2018-08-02.06:34:15.069>
actor = 'Eric.Wieser'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2014-09-26.21:36:35.226>
closer = 'yselivanov'
components = ['Library (Lib)']
creation = <Date 2007-07-31.10:16:40.000>
creator = 'michele_s'
dependencies = []
files = ['9384', '31902']
hgrepos = []
issue_num = 1764286
keywords = ['patch']
message_count = 12.0
messages = ['32574', '57779', '60062', '62177', '110708', '110752', '198575', '227654', '227655', '322728', '322775', '322918']
nosy_count = 10.0
nosy_names = ['michele_s', 'belopolsky', 'pitrou', 'christian.heimes', 'gpolo', 'ysj.ray', 'Claudiu.Popa', 'python-dev', 'yselivanov', 'Eric.Wieser']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1764286'
versions = ['Python 3.5']

[micheles]

Here is the issue:

$ cat example.py
import functools # I am using Python 2.5

def identity_dec(func):
    def wrapper(*args, **kw):
        return func(*args, **kw)
    return functools.update_wrapper(wrapper, func)

@identity_dec
def example(): 
    pass

>>> import inspect
>>> from example import example
>>> print inspect.getsource(example)
    def wrapper(*args, **kw):
        return func(*args, **kw)

You get the source code of the closure
and not what would be more meaningful, i.e.
the string

"""
@identity_dec
def example():
pass
"""

Of course one could argue that this is not a bug
(in a sense the inspect module is doing the right
thing) but still it is giving information which
is not very useful.

Looking at the guts of inspect.getsource, one discovers the origin
of the problem: inspect.findsource is looking at the attribute
.co_firstlineno of the decorated function code object.
Unfortunately .co_firstlineno is a read-only attribute, otherwise
it would be possibile to change functools.update_wrapper to set it to
the correct line number (i.e. the line where
the undecorated function is defined, -1). So
I don't think you can fix this in current
Python, but it is something to keep in mind for Python 2.6 and 3.0. It should also manage classmethods/
staticmethods and other decorators not implemented
as closures.

[tiran]

I'm setting the target to 2.6 and 3.0. Maybe somebody can come up with a
sensible patch.

[pitrou]

Rather than devising something specific to the co_firstlineno attribute,
why not have functools.update_wrapper add a "wrapped_func" attribute
pointing to the original function object? That way, each function
inspecting the decorated function would have the opportunity to walk the
decoration chain if it wants.

[gpolo]

I am attaching a patch that address this issue.

[abalkin]

I did not test the patch, but if it really addresses the issue, I don't see why such a simple fix would not be applied.

Guilherme,

Can you add unit tests? Also, the second chunk in your patch is just a style fix which is a distraction for reviewers.

[ysjray]

I don't think this patch is correct, because we don't know if the closure[0] is the wrapped function, consider the following case:

def fun():
    abc = 1
    def fun2():
        print(abc)
    print(inspect.getsource(fun2))

In this case, the __closure__ of fun2 is [cell(1), ], the patch doesn't work.

I think the behavior of inspect.getsource(arg) is no problem because it indeed gives the right source code of arg, no matter arg is a wrapper function or an original function. michele argues that the result is not meaningful, through in most case it seems reasonably because wrapper functions usually do litter valuable work, but it is not correct for all cases. What if a wrapper function does more valuable work than the wrapped function? In this case should getsouce() give the source code of wrapper function because it's more meaningful? The concept "wrapper" and "wrapped" should have no relation with its source code.

I suggest you assign a special named attribute of all wrapper functions to its wrapped function yourself, something like "wrapped_function", and then you can walk through the wrapper chain to find the real source you want for each wrapper function. Then the stdard library's update_wrapper() and getsource() can remain unchanged.

[PCManticore]

Hello. Attached patch which uses the new inspect.unwrap, added in http://hg.python.org/cpython/rev/2aa6c1e35b8a.

[python-dev]

New changeset ad9cc6124a19 by Yury Selivanov in branch 'default':
inspect: Fix getsource() to support decorated functions.
https://hg.python.org/cpython/rev/ad9cc6124a19

[1st1]

Thanks for the bug report and patch! Committed to 3.5.

[eric-wieser]

This now leaves inspect.getsource inconsistent with inspect.getsourcefile:

>> import inspect
>> from contextlib import contextmanager

>>> @contextmanager
... def func():
...    yield

>>> inspect.getsource(func)
'@contextmanager\ndef func():\n    yield\n'
>>>inspect.getsourcefile(func)
'C:\\Program Files\\Python 3.5\\lib\\contextlib.py'

Should getsourcefile be changed to match?

This is causing numpy/numpy#11639, but it's not clear if this is a bug or by design.

[1st1]

Should getsourcefile be changed to match?

I'd say yes. There's no point in getsourcefile returning the file location of the topmost decorator. Feel free to open a new issue and submit a PR to fix this!

[eric-wieser]

New issue opened at https://bugs.python.org/issue34305, along with a PR linked there.