New display system

[ellisonbg]

Hi,

This series of commits is a refactor and enhancement of the display system in IPython. This takes the idea of "display" present in sys.displayhook and generalizes it to handle different MIME types. The new display system has two parts:

• DisplayFormatter, which is responsible for taking a python object and producing a dict that contains the display data. This dict consists of key/value pairs, where the keys are MIME types and the values are the data for that MIME type. The DisplayFormatter uses a set of Formatter classes. Currently, we have formatters for text, html, svg, png, json, latex. The qtconsole only handles svg, png, html and plain text. Each of the formatters has a a special print method it looks for (pretty, html, etc.) as well as the pretty style API of printers.
• DisplayPublisher, which takes the dict produced by the DisplayFormatter and publishes it to all frontends. This is done using the PUB channel and a new message type "display_data".

There are two situations that trigger the display machinery to be triggered:

1. First, the IPython displayhook now calls these classes to do its main work.
2. There are now top-level functions in IPython.core.display that users can use to trigger the display of objects (display, display_html, etc.)

I have added display functions for matplotlib.figure.Figure which enables the displayhook and the display function to produce nice SVG output when fed a figure instance. In addition, the sympy profile can not pretty print and latex->png sympy expressions.

I will also post some nice qtconsole outputs that show examples of all of this.

[takluyver]

Amazing stuff! A couple of little points about docstrings:

• In core.formatters.DisplayFormatter.format, include and exclude are listed as returns in the docstring, while I think they should be under parameters.
• In core.formatters.PNGFormatter, the docstring mentions "define a :meth:__svg__ method", while I'm guessing it should be a __png__ method.

[ellisonbg]

Thanks for having a look. I just pushed fixes for these things.

[minrk]

This looks really great. I've looked over some of it, and I have a few notes:

core/display.py

in def display:
typo in docstring: exclue->exclude
why does display call InteractiveShell.instance() twice for each call? Why not pass an instance as the first argument, or make it an InteractiveShell method, or at least only call instance() just once?

core/displayhook.py

def compute_format_data:
docstring typos: method only compute the format... (computes)
and ... whose forat ... (format)

displaypub.py

in publish there is a print >>io.Term.cout, ...
since we depend on py2.6 now, do we want to use print_function in new code?

zmqshell.py

ZMQDisplayPublisher

replace:
msg = session.msg(...)
socket.send(msg)
with:
msg = session.send(socket, ...)

[ellisonbg]

Min, thanks for the comments. I have fixed these things. The display functions should remain functions as they are designed to be imported and used by users. I think this is a better approach than 1) polluting their namespace by default or 2) requiring them to do get_ipython first. I think we should probably move get_ipython to this model as well. I haven't fixed the session stuff, but will once your branch is merged and I rebase on top of that.

[minrk]

Okay, that makes sense.

You don't have to wait for my work to merge before making the change to the send pattern. The only changes I made have to do with being able to send a message twice, and tweaking the form of recv. That way, there's no reason for you to rebase, just use session.send(socket,...) instead of msg = session.msg(...)+socket.send_json(msg) in new sends, like you added in DisplayPublisher.

[fperez]

This looks great, excellent work!

Please go ahead and merge, once a few things (most fairly minor) are taken care of:

• [the only not totally trivial fix] as we chatted about, we should restore the part of pastefig() that did the logic for getting the figure objects with an easy to use function pulled in when --pylab is in use. The code was all there in the old pastefig(), we just need to pull it back, and adapt display() so it can take _args. Then the call can be made by the user as display(_getfigs()) or display(*getfigs(1, 3, f5)). If this proves to be a really common idiom, we can always (later) expose a 2-line convenience wrapper for it.
• in displaypub (and any new files) we should use "from future import print_function" and use print(). We're on 2.6 now and that will make the py3k branch cleaner. Backporting this fix to old files can be done as we go whenever it's easy/convenient, but for new files we should stick to the new syntax.
• L 74 in formatter.py, typo: immage -> image
• L 216 in displayhook.py, this code:

        format_dict = self.shell.display_formatter.format(result)
        return format_dict

can be changed to just

        return self.shell.display_formatter.format(result)

no need to create a variable just to return it.

Again, fabulous work. Thanks!!

[ellisonbg]

I wrote a getfigs function that does what you want and display now takes *args. I fixed everything else as well and have merged. Thanks for the reviews everyone!

[takluyver]

Just to add - from a Python 3 point of view, it's not a big deal whether you use print_function. 2to3 doesn't seem to have any trouble converting print statements to functions. Of course, it can handle it as a function as well, but don't lose too much sleep over converting them manually.