Support IPython interactive widgets used in Python code

[stevengj]

If I load something like vispy or Matplotlib's new nbagg backend, which displays in IPython using IPython's interactive-widget API, it would be good to have it work in Julia as well via PyCall and Interact.

What I think this would involve:

• Have a new PyInteract package (that requires PyCall, Interact, and IJulia).
• using PyInteract would pyimport("IPython.html.widgets"), and then monkey-patch it to remap some of the low-level function calls:
  • Creating a comm object in Python should instead call the corresponding IJulia/Interact routine
  • The on_msg callback function of that comm object should call back to IPython to handle the message on the Python side.

I don't know the internals of IPython at all, so some advice from @minrk, @fperez, @ellisonbg, and/or @Carreau would be very helpful:

• What is the minimal set of low-level IPython functions to monkey-patch in order to hook up IPython's widget code to the IJulia plumbing?
• Is monkey-patching the right thing here, or is there some better way to install hooks into IPython to change the widget plumbing?

cc: @shashi, @dcjones

[stevengj]

@jdfreder, it was suggested that you are the person to ask about this. Basically, I need to replace anything in the IPython widgets code that sends a zeromq message.

[stevengj]

@takluyver suggests that maybe I just need to patch the methods of the CommManager class in IPython/kernel/comm/manager.py.

[shashi]

IJulia's CommManager is independent of Reactive or Interact packages. I'd think if vispy could find a way to use CommManager to setup and send messages according to IPEP 23, this would start working without needing to go via Interact. One other way to do this could be hack IJulia to use PyCall to send messages by invoking IPython's code. This way, vispy would be doing the same, integrating right out of the box with IJulia. I'll try out vispy and see what I can do.

And wow, vispy can render WebGL. 👍 @SimonDanisch - you should try it out to get ideas for GLPlot.jl API.

[stevengj]

@shashi, I agree that Interact is actually irrelevant here, now that I look at it.

I definitely don't want to hack IJulia to send messages via Python. The hacking should go the other way around. Fortunately, I can hack IPython at runtime via monkey patching.

I will need to monkey-patch IPython's CommManager (in the IPython Python module) to call IJulia's equivalent calls (to open and close comm objects etcetera). Otherwise, Python will be sending comm messages to the wrong session and the wrong ZMQ socket. Also, it won't be hooked into IJulia's event loop.

From what I've heard here at EuroSciPy, vispy is indeed using IPEP 23, so hopefully everything should work once I patch IPython.

[fperez]

Needless to say @stevengj, let us know of what pushes you towards monkeypatching, so that we can consider how to do that properly upstream eventually...

[jdfreder]

Hi @stevengj , sorry I was on vacation the last week and the week before, I didn't catch this until now. @takluyver is right, just the methods of CommManager need to be patched, all the widget messages are forwarded through there.

[ranr01]

Hi @stevengj , Any progress with this issue?
If you'd like I'll be happy to try and tackle this if you give me some guidance.
I'm an experienced scientific programmer, and have a little bit of experience hacking the IPython notebook for a different project so I know a little bit about the architecture.
If you think it's a simple fix but just did not get to it I'd be happy to try and contribute.

[stevengj]

I haven't had a chance to work on it. I don't think it should be very complicated; at least, not much code should be needed. The main thing is to understand the plumbing of IPython vs. Interact.

[djsegal]

Any news on this? :(

[stevengj]

We (@shashi, @travigd, and I at JuliaCon 2019) have started to put together an implementation at https://github.com/JuliaPy/PyInteract.jl … still very much a work in progress, but this works: