Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Event callbacks which register/unregister other callbacks lead to surprising behavior #9447
tl;dr: the loop in this line isn't robust to
We ran into a funny situation in colaboratory, involving
ip = get_ipython() ev = ip.events def func1(*unused_args): print 'invoking func1' ev.unregister('post_run_cell', func1) ev.register('post_run_cell', func3) def func2(*unused_args): print 'invoking func2' ev.unregister('post_run_cell', func2) def func3(*unused_args): print 'invoking func3' ev.register('post_run_cell', func1) ev.register('post_run_cell', func2)
In principle, this should invoke the three functions in order. In reality, it invokes
The cause is easy to see after checking out this line:
The lesson is, of course, never mutate a list as you iterate over it.
I'm happy to send a PR, but first, I want to make sure we're on the same page about what the desired semantics are here.
I think we want to be flexible: given that the only exposed operations for callbacks are remove and append (and thankfully not reset), we can ensure
Other options all involve preventing this behavior. We could freeze state by instead making a copy of the list before we iterate over it, whence any modifications would only be visible on the next triggering of the event. Or we could add code to completely disallow modifications to the currently-executing list of callbacks for the duration of
Our use case
A common pattern we've used is "one-shot callbacks" -- a callback which, when invoked, removes itself from the list of callbacks. We use this for some forms of output manipulation, as well as occasionally to do one-time cleanup that needs to happen after the user's current code completes.
With the proposed solution, this is easy -- the body of a callback just includes a call to deregister itself. This currently works great, unless the one-shot callback is not the last callback in the list. (The example above was distilled from a variant of this -- a one-shot callback was registering another one-shot callback.) It also allows for forcing a callback to be idempotent -- register as many times as you'd like, and the first one can deregister any copies it finds.
With either of the other solutions, we're stuck basically inventing our own event system, and using it to handle cleanup.
Thanks for the thorough description!
In the abstract, I'd expect that newly registered handlers run the next time the event fires, but not for the event that's already being handled - i.e. we take a copy of the list of callbacks before running any of them.
That would still allow for a one-shot callback, I think - it would remove itself from the original list, and we'd continue iterating over the unchanged copy for the current event. It wouldn't allow one callback to remove another before it executes for the current event, but that seems like the kind of coupling that the event system shouldn't support: callbacks shouldn't depend on the order they're registered in.
Yes. That would probably the most natural behavior though if you notice the current implementation does exactly the opposite. It executes newly added callbacks but completely breaks down if one unregister during the callback.
If I were writing it from scratch I would probably go for that behavior but if you want to keep backwards compatibility one would need to do something more complicated.
Thanks Mark. I think I'm happy enough to say that one callback adding another to run on the same event is a corner case. I don't think the event system has seen much use yet, so I'd lean towards doing what makes most sense over striving to preserve compatibility.