Request: pickle-based pylab backend for ZMQ shell #2322

Open
tkf opened this Issue Aug 21, 2012 · 7 comments

Comments

Projects
None yet
4 participants
@tkf
Contributor

tkf commented Aug 21, 2012

It seems that matplotlib is going to support pickling figures (matplotlib/matplotlib#1020). Wouldn't it be nice to have a new pylab backend which sends png/svg figure and also pickled figure? This way, you can get inline figure in the notebook client but also GUI figure in the terminal client. You can even choose GUI backend to use at the client side.

I guess there will be a trouble when the ZMQ shell and client use different matplotlib version so you will need to detect incompatible version some how.

@pelson

This comment has been minimized.

Show comment
Hide comment
@pelson

pelson Aug 21, 2012

Contributor

It seems that matplotlib is going to support pickling figures

Yes. Though it should be said that it is pretty experimental (colorbars and legends are yet to work for instance). The ability to pickle the figure into a binary/ascii form for things such as the notebook was definitely one of my motivators. It should be said very explicitly here though: Pickled files are not secure. It is possible to inject arbitrary code into a pickle dump.. The most cited article on this can be found here.

Contributor

pelson commented Aug 21, 2012

It seems that matplotlib is going to support pickling figures

Yes. Though it should be said that it is pretty experimental (colorbars and legends are yet to work for instance). The ability to pickle the figure into a binary/ascii form for things such as the notebook was definitely one of my motivators. It should be said very explicitly here though: Pickled files are not secure. It is possible to inject arbitrary code into a pickle dump.. The most cited article on this can be found here.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Aug 21, 2012

Contributor

@pelson Thanks for the comment. To clarify, my suggestion is to use pickle for kernel-client messaging, not for saving figure. This won't bring security risk as long as you trust kernel, right? I think notebook client can simply discard pickle data since it is useless for Javascript.

Contributor

tkf commented Aug 21, 2012

@pelson Thanks for the comment. To clarify, my suggestion is to use pickle for kernel-client messaging, not for saving figure. This won't bring security risk as long as you trust kernel, right? I think notebook client can simply discard pickle data since it is useless for Javascript.

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Aug 21, 2012

Member

Interesting. So the main advantage would be being able to display
interactive figures from the "ipython console" and qtconsole frontends? Is
that worth the complexity? Both already have ways to display static views
of plots, and the commonest use cases for both are with a local kernel,
which can already display the interactive figures.

Also, if we do this, how would we avoid transmitting a large block of data
when we plot a big dataset? One of the advantages of using png by default
is that the data never gets very big, because hidden points needn't be
shown.

Member

takluyver commented Aug 21, 2012

Interesting. So the main advantage would be being able to display
interactive figures from the "ipython console" and qtconsole frontends? Is
that worth the complexity? Both already have ways to display static views
of plots, and the commonest use cases for both are with a local kernel,
which can already display the interactive figures.

Also, if we do this, how would we avoid transmitting a large block of data
when we plot a big dataset? One of the advantages of using png by default
is that the data never gets very big, because hidden points needn't be
shown.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Aug 21, 2012

Contributor

There were PR to support chaning backend (#2179) so I guess having interactive plot while having inline figure is a wanted feature. As matplotlib support backend switching only once and it does not work well when kernel is not local, I don't think switching backend is ultimate solution.

I am not sure how to solve problem with large figure. For notebook, you can filter it in notebook server. Is it possible (or planned) to client to "subscribe" to certain display type? If you can do that, then you can reduce unused transmission of large pickle data. Or you can expose it as magic command (something like %plotgui fig where fig is a figure object) so that user can have interactive figure only when s/he want.

Contributor

tkf commented Aug 21, 2012

There were PR to support chaning backend (#2179) so I guess having interactive plot while having inline figure is a wanted feature. As matplotlib support backend switching only once and it does not work well when kernel is not local, I don't think switching backend is ultimate solution.

I am not sure how to solve problem with large figure. For notebook, you can filter it in notebook server. Is it possible (or planned) to client to "subscribe" to certain display type? If you can do that, then you can reduce unused transmission of large pickle data. Or you can expose it as magic command (something like %plotgui fig where fig is a figure object) so that user can have interactive figure only when s/he want.

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Aug 21, 2012

Member

It's sent embedded in a JSON object, so I don't think there's a good way to
filter it out before it is sent to the client. I don't think the notebook
server parses the JSON, it just passes it through to the frontend. Some
sort of display negotiation, like HTTP 'Accept' headers might work - I'll
wait for someone who knows the architecture better to comment on that.

Member

takluyver commented Aug 21, 2012

It's sent embedded in a JSON object, so I don't think there's a good way to
filter it out before it is sent to the client. I don't think the notebook
server parses the JSON, it just passes it through to the frontend. Some
sort of display negotiation, like HTTP 'Accept' headers might work - I'll
wait for someone who knows the architecture better to comment on that.

@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Aug 22, 2012

Member

pickled objects should probably use the data_pub messages, which doesn't have to b64-encode for use in JSON messages. These messages also won't arrive at the notebook frontend, which doesn't support raw buffer data.

Member

minrk commented Aug 22, 2012

pickled objects should probably use the data_pub messages, which doesn't have to b64-encode for use in JSON messages. These messages also won't arrive at the notebook frontend, which doesn't support raw buffer data.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Aug 22, 2012

Contributor

Following advice from @minrk, supporting pickled figure in client side was pretty easy.

diff --git a/IPython/frontend/terminal/console/interactiveshell.py b/IPython/frontend/terminal/console/interactiveshell.py
index 1291822..9a40975 100644
--- a/IPython/frontend/terminal/console/interactiveshell.py
+++ b/IPython/frontend/terminal/console/interactiveshell.py
@@ -28,6 +28,7 @@
 from IPython.core import page
 from IPython.utils.warn import warn, error, fatal
 from IPython.utils import io
+from IPython.zmq.serialize import unserialize_object

 from IPython.frontend.terminal.interactiveshell import TerminalInteractiveShell
 from IPython.frontend.terminal.console.completer import ZMQCompleter
@@ -152,6 +153,15 @@ def handle_iopub(self):
                     if sub_msg["content"]["execution_state"] == "busy" :
                         pass

+                elif msg_type == 'data_message':
+                    if 'pickled_matplotlib' in sub_msg['content']['keys']:
+                        # Unserializing buffers adds figures to
+                        # matplotlib stack, so there is no need to
+                        # bind the unserialized object.
+                        unserialize_object(sub_msg['buffers'])
+                        from matplotlib import pyplot
+                        pyplot.show()
+
                 elif msg_type == 'stream' :
                     if sub_msg["content"]["name"] == "stdout":
                         print(sub_msg["content"]["data"], file=io.stdout, end="")

Then you can try it by running this in ipython console:

from IPython.zmq.datapub import publish_data
from matplotlib import pyplot

fig = pyplot.figure()
pyplot.plot([1, 2, 3, 0, 2])
publish_data({'pickled_matplotlib': fig})
Contributor

tkf commented Aug 22, 2012

Following advice from @minrk, supporting pickled figure in client side was pretty easy.

diff --git a/IPython/frontend/terminal/console/interactiveshell.py b/IPython/frontend/terminal/console/interactiveshell.py
index 1291822..9a40975 100644
--- a/IPython/frontend/terminal/console/interactiveshell.py
+++ b/IPython/frontend/terminal/console/interactiveshell.py
@@ -28,6 +28,7 @@
 from IPython.core import page
 from IPython.utils.warn import warn, error, fatal
 from IPython.utils import io
+from IPython.zmq.serialize import unserialize_object

 from IPython.frontend.terminal.interactiveshell import TerminalInteractiveShell
 from IPython.frontend.terminal.console.completer import ZMQCompleter
@@ -152,6 +153,15 @@ def handle_iopub(self):
                     if sub_msg["content"]["execution_state"] == "busy" :
                         pass

+                elif msg_type == 'data_message':
+                    if 'pickled_matplotlib' in sub_msg['content']['keys']:
+                        # Unserializing buffers adds figures to
+                        # matplotlib stack, so there is no need to
+                        # bind the unserialized object.
+                        unserialize_object(sub_msg['buffers'])
+                        from matplotlib import pyplot
+                        pyplot.show()
+
                 elif msg_type == 'stream' :
                     if sub_msg["content"]["name"] == "stdout":
                         print(sub_msg["content"]["data"], file=io.stdout, end="")

Then you can try it by running this in ipython console:

from IPython.zmq.datapub import publish_data
from matplotlib import pyplot

fig = pyplot.figure()
pyplot.plot([1, 2, 3, 0, 2])
publish_data({'pickled_matplotlib': fig})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment