Instance methods and WeakRefs don't mix. #58836
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = None closed_at = <Date 2012-11-17.18:01:46.023> created_at = <Date 2012-04-20.13:49:42.022> labels = ['type-feature', 'library'] title = "Instance methods and WeakRefs don't mix." updated_at = <Date 2012-11-26.17:46:13.363> user = 'https://bugs.python.org/Sundance'
activity = <Date 2012-11-26.17:46:13.363> actor = 'jcea' assignee = 'none' closed = True closed_date = <Date 2012-11-17.18:01:46.023> closer = 'pitrou' components = ['Library (Lib)'] creation = <Date 2012-04-20.13:49:42.022> creator = 'Sundance' dependencies =  files = ['25288', '27960'] hgrepos =  issue_num = 14631 keywords = ['patch'] message_count = 12.0 messages = ['158828', '158834', '175343', '175344', '175372', '175384', '175391', '175392', '175474', '175475', '175792', '175793'] nosy_count = 7.0 nosy_names = ['jcea', 'mark.dickinson', 'pitrou', 'daniel.urban', 'python-dev', 'Sundance', 'sfeltman'] pr_nums =  priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue14631' versions = ['Python 3.4']
The text was updated successfully, but these errors were encountered:
The behavior of instance methods makes it impossible to keep a weakref on them. The reference dies instantly even if the instance and the method still exist.
>>> def callback1(): ... print "In callback1." >>> class SomeClass: ... def callback2(self): ... print "In callback2."
>>> for callback in callbacks: ... callback() In callback1.
The WeakSet in the example above, and the weakref.ref() it employs, actually behave as specified. It's the particular nature of bound methods that causes the unexpected behavior.
From what I understand, instance methods are bound dynamically when looked up on the instance. A new object of type instancemethod is created each time:
>>> t1 = some_instance.callback >>> t2 = some_instance.callback >>> t1 is t2 False
So when a program calls weakref.ref(some_instance.callback), a new instancemethod object is created on the fly, a weakref to that object is created... and the instancemethod object dies, because its refcount is 0.
This fundamental difference between instance methods and other callables makes it painful to implement weakly referencing callback registries.
Changing the fundamental nature of instance methods to accommodate one single corner case doesn't seem worthwhile.
Similarly, changing the behavior of weakref.ref(), which does work as specified, is probably not a good idea.
My approach is thus to provide a new helper, WeakCallableRef, that behaves like weakref.ref(), but is safe to use on callables and does what you would naturally expect with instance methods.
It works by binding the lifetime of the ref to that of 1/ the instance bearing the method, and 2/ the unbound method itself. It is also safe to use on other function types beside instance methods, so implementations of callback registries don't have to special case depending on the callable type.
The unexpected behavior should also be mentioned somewhere in the weakref documentation, by the way.
See attached file for a proposed implementation.
Some more complex examples from various libraries:
I think both of these originated in pydispatcher.
The patch looks okay to me.
What does inheriting from 'ref' buy you? This feels a bit strange to me: the way I think of it, the WeakMethod *has* a weakref to the underlying object, rather than *being* a weakref to the underlying object. The __repr__ also seems a bit misleading as a result:
>>> o = Object() >>> m = o.some_method >>> WeakMethod(m) <weakref at 0x100665ae0; to 'Object' at 0x101115840>