New events system #5188

takluyver · 2014-02-21T20:02:09Z

As discussed in #5122, there's a need for a richer set of callbacks. This is the initial architecture and a few callbacks.

ip.callbacks is an instance of CallbackManager (I know @ivanov will love this name)
There is a set of known events, each with a name, e.g. 'pre_execute_explicit', and a signature, defined by a prototype function in IPython.core.callbacks.
At present, there's no checking of signatures, but I've deliberately designed things so we could use something like backcall to verify them.
Callbacks are registered and unregistered by passing the event name and the callback function. There's no explicit provision for unregistering a callback without a reference to the function, but you can clear all callbacks for a given event.
Our code calls ip.callbacks.fire('event_name', *args, **kwargs). Again, there's not yet any checking that the args passed match the specified signature, but there easily could be in the future.
Exceptions raised by callbacks are displayed, but do not prevent other callbacks from executing.

I've deprecated a couple of hooks in favour of callbacks. Our hooks system is designed for end users to customise things that shouldn't be repeated if they succeed - e.g. 'load this in an editor' or 'get text from the clipboard'. In contrast, callbacks are for extension code listening for particular events, so it should be possible to register several callbacks for the same event without them interfering with one another.

ellisonbg · 2014-02-21T20:18:53Z

Yeah! I haven't looked in detail, but I strongly think this is a great idea. Thanks so much for working on this. One comment about the naming of things. I agree that the callables being registered should be called "callbacks", but the actual abstraction is one of "events". This is almost the same exact design that we have/use in JavaScript and I think the naming of things should follow: events, trigger, on, rather than callbacks, fire and register. We also use the on style of things in Comms, Widgets, traitlets as well.

jdfreder · 2014-02-21T20:19:51Z

something like backcall to verify them.

Shameless self advertising 😉

takluyver · 2014-02-21T20:21:46Z

The best kind of self advertising ;-)

I'll look into giving things event-style names after lunch.

takluyver · 2014-02-25T21:42:12Z

OK, it took a bit longer, but I've used 'events' and 'trigger' in the names.

I've left 'register' instead of 'on', which I don't like. 'on' seems to brief and generic to be useful e.g. when searching, and doesn't have a neat counterpart like 'unregister': jquery's use of 'off' is cute but silly.

takluyver · 2014-02-27T22:19:22Z

As discussed in the dev meeting, I've added notes in the docs that this API is experimental for IPython 2.

takluyver · 2014-02-28T23:57:35Z

From discussion with @minrk and @ivanov , the event names are now:

pre/post_execute fire for all code execution triggered by a frontend - execute requests and comm messages.
pre/post_run_cell fire when user code is run - execute requests with silent=False.

(In the plain terminal case, both also fire when running user entered code)

ellisonbg · 2014-03-03T20:27:45Z

Needs rebase.

ellisonbg · 2014-03-03T20:28:41Z

IPython/core/events.py

+    """Manage a collection of events and a sequence of callbacks for each.
+
+    This is attached to :class:`~IPython.core.interactiveshell.InteractiveShell`
+    instances as a ``callbacks`` attribute.


as an events attribute

ellisonbg · 2014-03-03T20:29:29Z

IPython/core/events.py

+        self.callbacks = {n:[] for n in available_events}
+
+    def register(self, event, function):
+        """Register a new callback


a new event callback

…kers

ellisonbg · 2014-03-03T20:31:09Z

IPython/core/events.py

+# event_name -> prototype mapping
+available_events = {}
+
+def _collect(callback_proto):


I think the _collect name here is a bit vague. Maybe something like _define_event?

takluyver · 2014-03-03T20:33:22Z

Rebased and changed those things.

ellisonbg · 2014-03-03T20:34:05Z

IPython/core/events.py

+# No-op functions which describe the names of available events and the
+# signatures of callbacks for those events.
+# ------------------------------------------------------------------------------
+


To the reader who is not very familiar with how the internals of IPython work, I think the names of these events are still very confusing (pre_execute, post_execute, pre_run_cell, post_run_cell) and the descriptions don't really help. It is also not clear the order in which these are triggered. Even if we don't rename them we should add better comments to clarify when they are triggered.

We should also clarify how silent affects these things.

We had a long discussion about this over lunch on Friday, and those were the clearest set of names we could come up with. Any alternative suggestions will be duly considered.

possible clarifications:

execute events fire before/after any code might be executed (interactive code, silent execution, widget interaction, comm messages).

run_cell events fire only before/after explicit "regular" execution, e.g. shift-enter or interactive typing at the terminal.

For those interested in the internal implementation, this means that silent=False execution is the only circumstance to trigger run_cell events. All potential execution triggers the execute events, so when run_cell fires, execute will also fire.

I find the wording a bit tricky - after all, the internal kernel code is running all the time, so it's not before/after 'any' code runs. I want to call it 'user defined' code, but the frontend can send code for execution without the user writing it, as is the case in most silent execution.

I am fine with the names, but let's add a comment about how silent affects which are called.

Added a mention.

ellisonbg · 2014-03-03T20:37:46Z

IPython/core/interactiveshell.py

-            raise ValueError('argument %s must be callable' % func)
-        self._post_execute[func] = True
+        warn("ip.register_post_execute is deprecated, use "
+             "ip.callbacks.register('post_run_cell', func) instead.")


callbacks -> events like above.

ellisonbg · 2014-03-03T20:42:27Z

One non-inline question: do we need to update the pylab inline stuff to use this event system for rendering figures?

ellisonbg · 2014-03-03T20:44:42Z

See IPython.core.pylabtools.configure_inline_support for things that need to be updated. A bit hackish right now with manual walking of the hook data stucture :(

takluyver · 2014-03-03T20:48:52Z

Good point, that was actually what prompted these changes in the first place. I've switched it over.

takluyver · 2014-03-03T20:50:41Z

Oops, missed a bit. Give me a moment to amend that commit. Github views don't auto update quickly enough.

takluyver · 2014-03-03T20:56:24Z

OK, there we go. Less hackish now, although it's actually more lines of code in order to catch the exception. We should possibly consider having a pop-style API for unregister (i.e. ignore if not found). Or just wait for PEP 463 ;-)

minrk · 2014-03-03T22:37:43Z

IPython/core/events.py

+            try:
+                func(*args, **kwargs)
+            except Exception:
+                print("Error in callback {} (for {}):".format(func, event))


print to stderr, or no?

showtraceback() goes to stdout, looking at the implementation. Possibly it shouldn't, but this should go to the same stream, and changing showtraceback() is more invasive.

makes sense.

ellisonbg · 2014-03-03T23:12:21Z

I can confirm this fixes #5122 yeah!

minrk · 2014-03-04T17:39:38Z

IPython/core/interactiveshell.py

@@ -827,12 +836,21 @@ def set_hook(self,name,hook, priority = 50, str_key = None, re_key = None):

        setattr(self.hooks,name, dp)

+    #-------------------------------------------------------------------------
+    # Things related to callbacks


s/callbacks/events/

minrk · 2014-03-04T17:44:19Z

setupbase.py

                    _remove_startswith(requires, pkg)
                requires.append("gnureadline; sys.platform == 'darwin' and platform.python_implementation == 'CPython'")
                requires.append("pyreadline (>=2.0); sys.platform == 'win32' and platform.python_implementation == 'CPython'")
+                requires.append("mock; extra == 'test'; python_version < '3.3'")


let's test whether ; or and should separate extra and platform conditions. Hopefully these are not both correct.

tested, and ; fails with SyntaxError. This should be and.

minrk · 2014-03-04T17:52:00Z

Excellent, merging.

New events system

damianavila mentioned this pull request Feb 25, 2014

A simple extension to activate interactive support in the IPython notebook bokeh/bokeh#353

Merged

fperez added this to the 2.0 milestone Feb 27, 2014

ellisonbg reviewed Mar 3, 2014
View reviewed changes

Start of new callback system

e3e8fef

ellisonbg reviewed Mar 3, 2014
View reviewed changes

takluyver added 11 commits March 3, 2014 12:30

Add mock to requirements for testing

b4d5a82

Add docstrings for callbacks

823ffca

Cross link signatures docstrings

6b0a1bb

Add tests for callback infrastructure

e5a269a

Deprecate some hooks in favour of callbacks

5898f70

Use callbacks system for autoreload

934bce7

Document new callbacks system

6de7fab

Rename callbacks -> events (mostly), fire -> trigger

adb71c3

Add note about experimental API

1cca64b

Rename pre/post_execute_explicit events to pre/post_run_cell

49fa2c5

Rename shell_initialised for orthographically challenged English spea…

cb9d4aa

…kers

ellisonbg reviewed Mar 3, 2014
View reviewed changes

Clarity fixes pointed out by @ellisonbg

d56c61c

ellisonbg reviewed Mar 3, 2014
View reviewed changes

takluyver added 2 commits March 3, 2014 12:43

Update docstrings to refer to ip.events

13506a8

Use silly American spelling for initialised

bc20f65

callbacks -> events in docs

1b3e35d

Use new events API for inline backend flush_figures

25522b5

minrk reviewed Mar 3, 2014
View reviewed changes

Mention silent execution in docstrings

09e41c6

minrk reviewed Mar 4, 2014
View reviewed changes

Update comment

89392c3

minrk reviewed Mar 4, 2014
View reviewed changes

Fix wheel dependency specification

d39747e

minrk added a commit that referenced this pull request Mar 4, 2014

Merge pull request #5188 from takluyver/callbacks

4c8ff1d

New events system

minrk merged commit 4c8ff1d into ipython:master Mar 4, 2014

damianavila mentioned this pull request Mar 11, 2014

IPython Notebook Callback after a cell is returned #5317

Closed

This was referenced Mar 26, 2014

Weird interact behavior #5118

Closed

Call post execute hooks after processing widget msg #5122

Closed

Comm code is not run in the same context as notebook code #5006

Closed

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#5188 from takluyver/callbacks

c0e93b8

New events system

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New events system #5188

New events system #5188

takluyver commented Feb 21, 2014

ellisonbg commented Feb 21, 2014

jdfreder commented Feb 21, 2014

takluyver commented Feb 21, 2014

takluyver commented Feb 25, 2014

takluyver commented Feb 27, 2014

takluyver commented Feb 28, 2014

ellisonbg commented Mar 3, 2014

ellisonbg Mar 3, 2014

ellisonbg Mar 3, 2014

ellisonbg Mar 3, 2014

takluyver commented Mar 3, 2014

ellisonbg Mar 3, 2014

ellisonbg Mar 3, 2014

takluyver Mar 3, 2014

minrk Mar 3, 2014

takluyver Mar 3, 2014

ellisonbg Mar 3, 2014

takluyver Mar 3, 2014

ellisonbg Mar 3, 2014

takluyver Mar 3, 2014

ellisonbg commented Mar 3, 2014

ellisonbg commented Mar 3, 2014

takluyver commented Mar 3, 2014

takluyver commented Mar 3, 2014

takluyver commented Mar 3, 2014

minrk Mar 3, 2014

takluyver Mar 3, 2014

minrk Mar 4, 2014

ellisonbg commented Mar 3, 2014

minrk Mar 4, 2014

takluyver Mar 4, 2014

minrk Mar 4, 2014

minrk Mar 4, 2014

minrk commented Mar 4, 2014

New events system #5188

New events system #5188

Conversation

takluyver commented Feb 21, 2014

ellisonbg commented Feb 21, 2014

jdfreder commented Feb 21, 2014

takluyver commented Feb 21, 2014

takluyver commented Feb 25, 2014

takluyver commented Feb 27, 2014

takluyver commented Feb 28, 2014

ellisonbg commented Mar 3, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

takluyver commented Mar 3, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ellisonbg commented Mar 3, 2014

ellisonbg commented Mar 3, 2014

takluyver commented Mar 3, 2014

takluyver commented Mar 3, 2014

takluyver commented Mar 3, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ellisonbg commented Mar 3, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minrk commented Mar 4, 2014