Add inspect.unwrap(f) to easily unravel "__wrapped__" chains

[ncoghlan]

BPO	13266
Nosy	@rhettinger, @jcea, @ncoghlan, @ezio-melotti, @merwok, @florentx, @meadori, @durban
Dependencies	bpo-17482: functools.update_wrapper mishandles wrapped
Files	inspect_unwrap.patch: inspect.unwrap - 1st patch p13266-2.diff: Updated patch for 3.4 with documentation inspect_unwrap_3.patch issue13266_inspect_unwrap_4.diff: Patch with ability to stop unwrapping early

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2013-07-28.10:00:30.485>
created_at = <Date 2011-10-26.06:43:11.307>
labels = ['type-feature', 'library']
title = 'Add inspect.unwrap(f) to easily unravel "__wrapped__" chains'
updated_at = <Date 2013-07-28.10:00:30.484>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2013-07-28.10:00:30.484>
actor = 'python-dev'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2013-07-28.10:00:30.485>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2011-10-26.06:43:11.307>
creator = 'ncoghlan'
dependencies = ['17482']
files = ['25368', '27790', '27876', '30927']
hgrepos = []
issue_num = 13266
keywords = ['patch']
message_count = 9.0
messages = ['146420', '146421', '159336', '174179', '174767', '184655', '193095', '193165', '193814']
nosy_count = 10.0
nosy_names = ['rhettinger', 'jcea', 'ncoghlan', 'ezio.melotti', 'eric.araujo', 'flox', 'meador.inge', 'daniel.urban', 'python-dev', 'aliles']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue13266'
versions = ['Python 3.4']

[ncoghlan]

I just got bitten by the singularly unhelpful results of doing inspect.getsource(generator_context_manager).

Now that @functools.wraps adds the __wrapped__ attribute, perhaps inspect.getsource(f) should follow the wrapper chain by default?

This would affect other inspect APIs as well, such as getgeneratorstate() and the new getclosurevars() and getgeneratorlocals() that will be available in 3.3

While the source code of the wrapper could still be accessed by doing inspect.getsource(f.__code__), this isn't a reliable alternative in general (e.g. it doesn't work when introspecting closure state, since that needs access to the function object to look things up correctly). Accordingly, there would need to be a way to disable the "follow wrapper chains behaviour".

Alternatively (and more simply), we could just add an "inspect.unwrap(f)" function that followed a chain of __wrapped__ attributes and returned the last object in the chain (using an "already seen" set to avoid hit the recursion limit if someone sets up a wrapper loop). Applying this would then be a manual process when the initial output of an inspect function is clearly coming from a wrapper rather than the underlying function definition.

[ncoghlan]

After a little thought, I think the explicit "unwrap" function is the only viable approach. Doing the unwrapping implicitly just has too many nasty corner cases to track down to ensure we aren't losing existing functionality.

I'd also suggest that we add a __wrapped__ alias for the 'func' attribute of functools.partial objects.

[durban]

I've attached a patch implementing inspect.unwrap (+ some tests).

[aliles]

I've updated the patch for the current default branch (to be Python 3.4) and added documentation to the inspect module for the new unwraps function. Functionally unwraps and it's tests are unchanged.

[durban]

I've attached a patch addressing the comments on Rietveld. I've added another modification: inspect.signature uses inspect.unwrap. (It already tried to unwrap the function, but it wasn't protected from infinite recursion. I don't know if this worth fixing in 3.3.)

[ncoghlan]

Turns out there's a bug in the implementation of functools.update_wrapper :P

Added that as a dependency, since this API doesn't make sense until update_wrapper is behaving itself.

The new tests didn't pick it up because they don't use wraps or update_wrapper, they set __wrapped__ directly.

Also, the replacement of the recursion in inspect.signature is incorrect - we want to interleave checks for __signature__ as we recurse through the stack of wrapper functions.

[ncoghlan]

Added a version that allows the iteration to be terminated early if certain criteria are met, which is needed for a robust implementation of inspect.signature.

However, I'm thinking the callback based approach in this version isn't especially Pythonic, so I'm thinking it may be better to change the API to a generator function. That way the iterator can still take care of the wrapper loop detection, without needing the clumsy predicate API for early termination.

Instead, you would just use an ordinary search loop, and if you wanted the innermost function unconditionally you could do something like:

    for f in functools.unwrap(original): pass
    # f is now the innermost function

[aliles]

My +1 is for the callback based approach. The brevity of the search loop for finding the innermost function is (in my opinion at least) non-obvious, relying on for loops not having their own scope as it does.

If a generator based API was adopted instead, I propose a convenience function (unwrap_all?) to help developers avoid writing code like:

    inner = None
    for inner in functools.unwrap(outer):
        pass
    if inner is None:
        inner = outer

Which combines a misunderstanding of the API with for loop scope shortcut.

[python-dev]

New changeset 2aa6c1e35b8a by Nick Coghlan in branch 'default':
Close bpo-13266: Add inspect.unwrap
http://hg.python.org/cpython/rev/2aa6c1e35b8a