-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Event callbacks which register/unregister other callbacks lead to surprising behavior #9447
Comments
Thanks for the thorough description! In the abstract, I'd expect that newly registered handlers run the next time the event fires, but not for the event that's already being handled - i.e. we take a copy of the list of callbacks before running any of them. That would still allow for a one-shot callback, I think - it would remove itself from the original list, and we'd continue iterating over the unchanged copy for the current event. It wouldn't allow one callback to remove another before it executes for the current event, but that seems like the kind of coupling that the event system shouldn't support: callbacks shouldn't depend on the order they're registered in. |
Yes. That would probably the most natural behavior though if you notice the current implementation does exactly the opposite. It executes newly added callbacks but completely breaks down if one unregister during the callback. If I were writing it from scratch I would probably go for that behavior but if you want to keep backwards compatibility one would need to do something more complicated. |
FTR: We are happy to adopt Thomas's semantic, (of freezing the list) |
Thanks Mark. I think I'm happy enough to say that one callback adding another to run on the same event is a corner case. I don't think the event system has seen much use yet, so I'd lean towards doing what makes most sense over striving to preserve compatibility. |
Event callbacks can register or unregister new callbacks for the same event while executing, and the previous triggering implementation allowed for event callbacks to be inadvertently skipped. The fix is to make a copy of the list of callbacks before executing any of them. With this change, the resulting semantics are simple: any callbacks registered before triggering are executed, and any new callbacks registered are only visible at the next triggering of the event. Note that this could potentially break existing callers who expected newly-appended callbacks were immediately executed. Fixes ipython#9447. Originally based on a patch by @marksandler2. working on event callbacks
Event callbacks can register or unregister new callbacks for the same event while executing, and the previous triggering implementation allowed for event callbacks to be inadvertently skipped. The fix is to make a copy of the list of callbacks before executing any of them. With this change, the resulting semantics are simple: any callbacks registered before triggering are executed, and any new callbacks registered are only visible at the next triggering of the event. Note that this could potentially break existing callers who expected newly-appended callbacks were immediately executed. Fixes ipython#9447. Originally based on a patch by @marksandler2.
Event callbacks can register or unregister new callbacks for the same event while executing, and the previous triggering implementation allowed for event callbacks to be inadvertently skipped. The fix is to make a copy of the list of callbacks before executing any of them. With this change, the resulting semantics are simple: any callbacks registered before triggering are executed, and any new callbacks registered are only visible at the next triggering of the event. Note that this could potentially break existing callers who expected newly-appended callbacks were immediately executed. Fixes ipython#9447. Originally based on a patch by @marksandler2.
tl;dr: the loop in this line isn't robust to
register
/unregister
. What's the desired behavior?The problem
We ran into a funny situation in colaboratory, involving
post_run_cell
callbacks which need to potentially add or remove otherpost_run_cell
callbacks. Here's a really simplified version:In principle, this should invoke the three functions in order. In reality, it invokes
func1
andfunc3
, but notfunc2
. Even worse, at the end of this cell's execution, the list of registered callbacks is[func2, func3]
, which is not the list of callbacks we saw execute. Sofunc2
, the only function that stays in the list of callbacks the whole time, is the only one we don't execute. 😉The cause is easy to see after checking out this line:
func1
is our callback, and[func1, func2]
is our list of callbacks.func1
executes, and our list of callbacks is now[func2, func3]
.func3
. unfortunately, we've now skippedfunc2
. sadface.The lesson is, of course, never mutate a list as you iterate over it.
Potential solutions
I'm happy to send a PR, but first, I want to make sure we're on the same page about what the desired semantics are here.
I think we want to be flexible: given that the only exposed operations for callbacks are remove and append (and thankfully not reset), we can ensure
Other options all involve preventing this behavior. We could freeze state by instead making a copy of the list before we iterate over it, whence any modifications would only be visible on the next triggering of the event. Or we could add code to completely disallow modifications to the currently-executing list of callbacks for the duration of
trigger
, meaning a user would get an error if they tried.Our use case
A common pattern we've used is "one-shot callbacks" -- a callback which, when invoked, removes itself from the list of callbacks. We use this for some forms of output manipulation, as well as occasionally to do one-time cleanup that needs to happen after the user's current code completes.
With the proposed solution, this is easy -- the body of a callback just includes a call to deregister itself. This currently works great, unless the one-shot callback is not the last callback in the list. (The example above was distilled from a variant of this -- a one-shot callback was registering another one-shot callback.) It also allows for forcing a callback to be idempotent -- register as many times as you'd like, and the first one can deregister any copies it finds.
With either of the other solutions, we're stuck basically inventing our own event system, and using it to handle cleanup.
The text was updated successfully, but these errors were encountered: