Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPEP 21: widget messages #4195

Merged
merged 47 commits into from Oct 24, 2013
Merged

IPEP 21: widget messages #4195

merged 47 commits into from Oct 24, 2013

Conversation

minrk
Copy link
Member

@minrk minrk commented Sep 11, 2013

IPEP 21 has full description of the proposal.

This is just the basic communication tube, through which widgets might communicate.

It's pretty basic, but it seems to work reasonably well.

Some example use cases for the Comms implemented here: http://nbviewer.ipython.org/6547109

This also refactors callbacks in a clearer way, allowing callback cleanup since widget messages dramatically exacerbate #2524.

closes #2524.

@stevengj
Copy link

cc: @dcjones, who has been implementing JS-based plotting widgets in IJulia.

Would be nice in the PEP to refer to the "kernel" rather than "Python" side, and to separate the things that any kernel must do from the specific implementation details in the IPython kernel.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

Would be nice in the PEP to refer to the "kernel" rather than "Python" side

I tried to do this, I will read through again and see where I slipped up. There are actually three pieces here:

  1. message spec (Kernel and Frontend)
  2. Python API implementing the spec
  3. Javascript API implementing the spec

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

I read through the IPEP, and Python is never mentioned in the spec - I only use Kernel and Frontend there. When I switch gears to talk about the specific APIs of the Python and Javascript implementations, I discuss the languages in which those implementations exist. I have clarified the transition, though.

@ellisonbg
Copy link
Member

Great work, I love the symmetry and simplicity!

Widget Creation:

  • Should clarify how the widget id is selected - kernel picks.
  • Do the two sides share the widget id?
  • Clarify what the frontend should do when it receives the widget_create message
  • What happens with the frontend gets a widget_create message but doesn't know about that widget type? Seems like the frontend should send a widget_destroy message.
  • Do we allow the frontend to send the widget_create message?

Widget object:

  • I think we should try the simple design - just have data and not the full message. The data part is general enough to really cover everything. The only case where I could imagine we would want to include the metadata is if we started to define the details of the data JSON object to include things like our display messages.

@ellisonbg
Copy link
Member

I think the biggest question I have at this point is how widgets relate to our standard display messages. What happens if a widget needs to update the state of a matplotlib plot? Seems like there needs to be some coordination there. Do widgets handle the display messages? Do why manage their own OutputArea?

Another question is if we are going to have any notion of widget nesting. If there is a plot that updates with the values of 4 sliders and 2 checkboxes, is that 1 widget or 6?

var msg_types = ['widget_create', 'widget_destroy', 'widget_update'];
for (var i = 0; i < msg_types.length; i++) {
var msg_type = msg_types[i];
kernel.register_iopub_handler(msg_type, $.proxy(this[msg_type], this));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we make sure that we unregister these handlers when the widget is destroyed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are only registered once - there is only one WidgetManager.

@ellisonbg
Copy link
Member

In the IPEP, we should emphasize that the widget_create and widget_destroy messages mean that the receiver should do those actions. Part of what is a bit confusing with a fully symmetric protocol is preventing infinite loops of messages. To prevent this, we should be very clear that the handle methods should never send the same message type it is handling.

@ellisonbg
Copy link
Member

Could you point to your examples?

@ellisonbg
Copy link
Member

@ivanov and @jdfreder I think the best way for us to test this widget messages is to use it to write our higher level JS/Python APIs. You are the two leads on that (Paul on the Python side, Jon on the Javascript side). Can you both start to think about that stuff. I am more than will to have design discussion with both of you. I want us to start this stuff as soon as possible so we have time to iterate and bake the design before 2.0.

@jdfreder
Copy link
Member

@ellisonbg sounds like a good idea. When are you going to be back in town?

@jasongrout
Copy link
Member

In the PEP, you ask: "Question: the handle_foo messages get the data dict, not the full message. This means they don't have access to metadata, etc. Should the handlers get the full message instead?"

I think yes. It would be useful, for example, to access the metadata. It might also be useful to see the timestamp when a message was generated, for example. And at least passing in the metadata would be more consistent with other handlers, right?

@ellisonbg
Copy link
Member

@jdfreder I will be back Sat night, back in the office on Monday. I will be in touch.

@ellisonbg
Copy link
Member

@jasongrout makes a good point. I could imagine that for some widgets, the timestamps are important - they could be used to throttle the update rates.

@ellisonbg
Copy link
Member

I should mention - I play some last week with some slider widgets and quickly found that it was easy to flood the kernel with calls. I am wondering if we want to add code on the JS side that can automatically throttle the rate at which the update messages are send. I think this is going to be a really common problem that most highly interactive widgets have.

@jasongrout
Copy link
Member

Another question to consider: How do you lay out multiple widgets? Suppose I want two sliders next to each other, above an interactive plot, with two buttons off to the side? I can construct the necessary html, for example. How do I specify where to put the widgets in the html? This is related to @ellisonbg's comment about nested widgets.

For Sage interacts, we have 3 layout systems, two of which are currently active. One is a list of lists (i.e., rows of columns). Another that might be more applicable here is to just spit out the html, then give the "locations", which are jquery selectors for the places to put the widgets.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

An update from the dev meeting and today's discussion:

These are not widgets, they are the basic communication object used to implement widgets. As a result, they should not be called widgets, but something else. I think I'm going with Comm for now. When you create a Widget, it will use a Comm, but it will not be a Comm.

Widget Creation:

Should clarify how the widget id is selected - kernel picks.

The creator picks the ID, can be done from either side.

Do the two sides share the widget id?

Yes

Clarify what the frontend should do when it receives the widget_create message
What happens with the frontend gets a widget_create message but doesn't know about that widget type?
Seems like the frontend should send a widget_destroy message.

Right now, it just logs an error. widget_destroy isis probably the right thing to do.

Do we allow the frontend to send the widget_create message?

Yes, the spec is fully symmetrical. I initially didn't allow this, because of a note from the dev meeting,
but it is actually more complicated to add the asymmetry.

Widget object:

I think yes. It would be useful, for example, to access the metadata.

Okay, I will pass along the whole message - it's not appreciably more complicated.

In the IPEP, we should emphasize that the widget_create and widget_destroy messages mean that the receiver should do those actions.

Will do.

Part of what is a bit confusing with a fully symmetric protocol is preventing infinite loops of messages. To prevent this, we should be very clear that the handle methods should never send the same message type it is handling.

I don't think so. Other than creation / destruction, there is only one message type, so if message handlers should ever send a message back at all, this restriction cannot be enforced. It makes perfect sense for an update from one side to result in a different update coming back (e.g. changing a value causing a replot). It is up to the implementer to ensure they don't get stuck.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

@jasongrout - this is not GUI widgets, only the communication part. Layout, etc. is entirely independent of this.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

I play some last week with some slider widgets and quickly found that it was easy to flood the kernel with calls. I am wondering if we want to add code on the JS side that can automatically throttle the rate at which the update messages are send. I think this is going to be a really common problem that most highly interactive widgets have.

Yes, I ran into this myself, and quickly started throttling requests. I'm not sure how to do it in a generic way that we can reasonably enforce on all widget authors, though.

@ellisonbg
Copy link
Member

@minrk this brings up a good point about naming. The current naming
implies that there will be a 1-to-1 mapping between Widget objects and UI
controls, but that is definitely not the case. A single widget object
could manage 100 UI controls. Will have to think more about the naming of
things.

On Thu, Sep 12, 2013 at 2:53 PM, Min RK notifications@github.com wrote:

@jasongrout https://github.com/jasongrout - this is not GUI widgets,
only the communication part. Layout, etc. is entirely independent of this.


Reply to this email directly or view it on GitHubhttps://github.com//pull/4195#issuecomment-24359014
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@jasongrout
Copy link
Member

@minrk: it makes sense that layout issues are above the communication layer. But the creation handler will have to address this somehow, so that's another vote for the whole message (including maybe layout metadata) being passed into the create handler.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

I think the biggest question I have at this point is how widgets relate to our standard display messages. What happens if a widget needs to update the state of a matplotlib plot? Seems like there needs to be some coordination there. Do widgets handle the display messages? Do why manage their own OutputArea?

That seems like it would be up to the Widget implementation - some might hook up events on an existing OutputArea, some might manipulate their own div elsewhere (sidebar namespace browser). It doesn't seem like a question that can have a generic answer.

Another question is if we are going to have any notion of widget nesting. If there is a plot that updates with the values of 4 sliders and 2 checkboxes, is that 1 widget or 6?

I don't know. I also am not sure this has any bearing on the code here - this is just the very basic communication code for allowing some object in the Kernel to talk to some object in the Frontend and vice versa.

@jasongrout
Copy link
Member

@minrk: if you're changing the names to reflect a message channel, maybe instead of update it should be send.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

@jasongrout reading my mind - I have switched create/update/destroy to more comm-appropriate open/send/close.

@jasongrout
Copy link
Member

I like the simplicity of the open/send/close/handlers idea. Basically you are just creating a private message channel on top of the existing messaging framework. I can see how just passing the data attribute makes sense---the IPython message wrapping the data attribute is nothing but routing information.

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

I can see how just passing the data attribute makes sense---the IPython message wrapping the data attribute is nothing but routing information.

That's true - and if the entity constructing the object is interesting in timings, it can always add them to the data messages. I guess I will live it using just data for now, but we can think a bit more carefully later on about handing off the whole message. It is not at all a difficult change to make.

@jasongrout
Copy link
Member

So how do you handle a channel open message from a frontend to a kernel that has multiple frontends? Do you broadcast out an open message to all other frontends (like you would echo input)? What about channel close messages? Are send messages from the kernel sent to all frontends?

@minrk
Copy link
Member Author

minrk commented Sep 12, 2013

So how do you handle a channel open message from a frontend to a kernel that has multiple frontends? Do you broadcast out an open message to all other frontends (like you would echo input)? What about channel close messages? Are send messages from the kernel sent to all frontends?

This is a good point, where there is an asymmetry - one kernel, possibly multiple frontends. It seems to me that it makes sense in general for a Comm on the Kernel side to re-broadcast both open and destroy messages that it has received. Then the only handling needed on the Frontend-side is to ignore open and close messages for Comms it has already opened or closed. I don't think that same generalization can be made for the 'normal' messages in between open and close, and that decision should be left to widget implementers.

* @submodule comm
*/

var IPython = (function (IPython) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to start using require ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to use require but that should be a separate PR.

@ellisonbg
Copy link
Member

A few more thoughts:

  • I thought about the name more and I am liking the name Channel instead of Comm. I like the open, close, send names though.
  • With our renames, these classes really become focused on communications. However, the handle_send method will presumably have all of the view logic for our "widgets" (here I am using that name to include the frontend UI side of things as well. IOW, I think it is important to cleanly separate the model from the view, but with the current design, the handle_send method would have the view logic.
  • I think it is important to have an object that is bound to a particular cell and has a handle on that cells DOM elements for view manipulation. Is that object the Comm/Channel object? Is that a new View object? Are Comm/Channel objects bound to cells?

@minrk
Copy link
Member Author

minrk commented Oct 23, 2013

I think this is ready to go, if anyone wants to have a last look.

@jasongrout
Copy link
Member

+1 to merging.

minrk added a commit that referenced this pull request Oct 24, 2013
@minrk minrk merged commit de09adf into ipython:master Oct 24, 2013
@jasongrout
Copy link
Member

Woohoo! Great work!

@ellisonbg
Copy link
Member

Awesome!

On Thu, Oct 24, 2013 at 12:55 PM, Jason Grout notifications@github.comwrote:

Woohoo! Great work!


Reply to this email directly or view it on GitHubhttps://github.com//pull/4195#issuecomment-27024534
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@SylvainCorlay
Copy link
Member

Great. Thank you.

@fperez
Copy link
Member

fperez commented Oct 24, 2013

Fantastic job, guys. Very, very happy about this.

@Carreau
Copy link
Member

Carreau commented Oct 26, 2013

@minrk in this, kernel.js you changed object_info_request to object_info. it broke tooltip, and is an API change (imho) what should we do about ?

(I should have seen it, sorry)

Carreau added a commit to Carreau/ipython that referenced this pull request Oct 26, 2013
comm merging renamed object_info_request to object_info
@minrk
Copy link
Member Author

minrk commented Oct 26, 2013

Must have been mixed up as part of the rebase, I know I tested tooltips. It was wrong to use _request in one method name, where we leave it off of the others (execute, complete).

@minrk minrk deleted the widget-msg branch October 26, 2013 17:44
mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
comm merging renamed object_info_request to object_info
brandoncurtis pushed a commit to brandoncurtis/sagecell that referenced this pull request Jan 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Notebook] Clear kernel queue
9 participants