Introspection generator and function closure state #57271
Comments
|
Based on the python-ideas thread about closures, I realised there are two features the inspect module could offer to greatly simplify some aspects of testing closure and generator behaviour: inspect.getclosure(func)
Returns a dictionary mapping closure references from the supplied function to their current values.
inspect.getgeneratorlocals(generator)
Returns the same result as would be reported by calling locals() in the generator's frame of executionThe former would just involve syncing up the names on the code object with the cell references on the function object, while the latter would be equivalent to doing generator.gi_frame.f_locals with some nice error checking for when the generator's frame is already gone (or the supplied object isn't a generator iterator). |
|
I'll take a shot and writing a patch for this one. Nick, are the >>> def make_adder(x):
... def add(y):
... return x + y
... return add
...
>>> def curry(func, arg1):
... return lambda arg2: func(arg1, arg2)
...
>>> def less_than(a, b):
... return a < b
...
>>> greater_than_five = curry(less_than, 5)
>>> def closure(func):
... vars = [var for var in func.__code__.co_freevars]
... values = [cell.cell_contents for cell in func.__closure__]
... return dict(zip(vars, values))
...
>>> inc = make_adder(1)
>>> print(closure(inc))
{'x': 1}
>>> print(closure(greater_than_five))
{'arg1': 5, 'func': <function less_than at 0xb74c6924>}? |
|
See: 454 /* func_new() maintains the following invariants for closures. The |
|
Yep, that looks right to me. The eval loop then references those cells from the frame object during execution. |
|
Huh, I didn't actually realise getclosure() could be written as a one liner until seeing Meador's version above: {var : cell.cell_contents for var, cell in zip(func.__code__.co_freevars, func.__closure__)} |
|
Here is a first cut at a patch. There is one slight deviation from the original spec:
The attached patch returns empty mappings for these cases. I can easily |
|
Because a generator can legitimately have no locals: >>> def gen():
... yield 1
...
>>> g = gen()
>>> g.gi_frame.f_locals
{}Errors should be reported as exceptions - AttributeError or TypeError if there's no gi_frame and then ValueError or RuntimeError if gi_frame is None. |
|
The function case is simpler - AttributeError or TypeError if there's no __closure__ attribute, empty mapping if there's no closure. I've also changed my mind on the "no frame" generator case - since that mapping will evolve over time as the generator executes anyway, the empty mapping accurately reflects the "no locals currently defined" that applies when the generator either hasn't been started yet or has finished. People can use getgeneratorstate() to find that information if they need to know. |
|
Here is an updated patch with error handling. One other thought is that |
|
No, the naming problem had occurred to me as well. Given the 'vars' builtin, perhaps 'getclosurevars' would do as the name? |
I like vars. Updated patch attached. |
|
In reviewing Meador's patch (which otherwise looks pretty good), I had a thought about the functionality and signature of getclosurevars(). Currently, it equates "closure" to "nonlocal scope", which isn't really true - the function's closure is really the current binding of *all* of its free variables, and that includes globals and builtins in addition to the lexically scoped variables from outer scopes. So what do people think about this signature: ClosureVars = namedtuple("ClosureVars", "nonlocals globals builtins unbound")
def getclosurevars(func):
"""Returns a named tuple of dictionaries of the current nonlocal, global and builtin references as seen by the body of the function. A final set of unbound names is also provided."""
# figure out nonlocal_vars (current impl)
# figure out global_vars (try looking up names in f_globals)
# figure out builtin_vars (try looking up names in builtins)
# any leftover names go in unbound_vars
return ClosureVars(nonlocal_vars, global_vars, builtin_vars, unbound_vars)Also, something that just occurred to me is that getclosurevars() should work for already instantiated generator iterators as well as generator functions, so the current typecheck may need to be made a bit more flexible. |
|
Nick, the revised definition of 'getclosurevars' seems reasonable to me. |
|
I didn't get around to updating my patch with Nick's comments yet. Nick, the v3 patch I have attached still applies. I am happy to update it per your comments (promptly this time) or you can take it over. Whichever. |
|
Meador: I probably won't get to this until the weekend, so go ahead and update the patch if you have time. |
|
Attached patch implements both new functions, but I'm going to drop getgeneratorlocals for now and move that idea to a new issue. |
|
I created bpo-15153 to cover getgeneratorlocals. Attached patch is just for record keeping purposes - I'll be committing this change shortly. |
|
New changeset 487fe648de56 by Nick Coghlan in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: