Consider integrating https://github.com/tommikaikkonen/prettyprinter

[njsmith]

This library is very young, but looks really excellent: https://tommikaikkonen.github.io/introducing-prettyprinter-for-python/

Compared to IPython's current pretty-printing system, this adds syntax-highlighting, what sounds like a better layout algorithm, and a much simpler system for customization.

For example, here's stock IPython:

And here's IPython with prettyprinter integration (i.e. monkeypatching) enabled (note that this uses a different set of highlight colors, whoops, but you get the idea):

And here's the kind of hoop-jumping I had to do to integrate patsy with IPython's current pretty-printing system: https://github.com/pydata/patsy/blob/924b387652d13f816fd4a32983aae2ccba38fd84/patsy/util.py#L437-L454
That's a helper function to generate reprs like Class(arg1, arg2, kwarg=something). I'm not sure anyone else ever took the trouble to actually implement this in their library. OTOH in prettyprinter, this is available as a trivial built-in.

I'm sure there are a number of obstacles here; off the top of my head:

• prettyprinter currently requires 3.6+, while IIRC even the next release IPython will still support at least 3.5

• there'd need to be some backcompat story for code that does use _repr_pretty_ and all that (I guess? I don't know how much of this code is out there)

• there'd need to be some story about how this works in the notebook (does the notebook understand ANSI color? do we need an HTML output option?)

• Does @tommikaikkonen actually want the responsibility of having Python's userbase descend upon their innocent little library

• probably other things I'm missing

But I figured I'd at least raise the issue here so y'all were aware this existed and could at least start the conversation.

[tommikaikkonen]

Thanks @njsmith for the kind words! IPython is great, and I've been using prettyprinter daily with it for a while now, and am really enjoying the experience. Here are my thoughts on the points you raised.

• There isn't much work to do to get the package working on Python 3.5. I had to remove Python 3.6 features to get the docs to build on ReadTheDocs.io (f-strings and use of enum.auto), so it probably runs on Python 3.5, but I haven't confirmed.

• Implementing output to HTML or other media would be fairly straightforward. The code in prettyprinter that writes the final output to a stream, a renderer, takes as input an iterable of SDocs (Simple Docs), which can be text, a newline, or the start or end of an annotation, so it's fairly agnostic to the output medium. Here's the function that renders colored output: https://github.com/tommikaikkonen/prettyprinter/blob/master/prettyprinter/color.py#L175-L227

• Replacing the default IPython pretty printer would be a bit early right now, as it was just released to the public, but after the library gets some real-life usage and works out the initial kinks, I think it could be a good fit.

  To elaborate about those kinks, the major difference to pprint and IPython.lib.pretty is that prettyprinter is not a streaming pretty printer. It builds a full intermediate representation of the output, a tree of "Docs" that represents all possible layouts of the data, before running the layout algorithm to choose the best one. This enables a lot of nice things internally, but without proper safeguards (e.g. truncating long sequences), this will cause very large data to hang for a while before anything shows up on screen. In addition to truncation, there are some easy pickings for performance optimization in the library that will alleviate this. These points should be fixed before considering full integration.

• Supporting legacy _repr_pretty_ should be possible. There are some edge cases where choosing between a printer registered with prettyprinter and _repr_pretty_ would be tough. It may also be possible for prettyprinter to support _repr_pretty_ implementations directly through a compatibility layer, since they are based on the same layout primitives, but I haven't explored that.

• As for the responsibility of having a big user base—from what I can tell, the majority of IPython.lib.pretty, especially its layout API and core, have been quite stable for a long time. I predict the same for prettyprinter. The layout primitives and algorithm are not novel for the most part, and are proven to work in other libraries. Most of the maintenance likely revolves around the public API, integrations and pretty printer definitions for standard library types, none of which require a deep understanding of the internals. It shouldn't be too hard for other people to contribute and eventually take over maintenance, just like IPython.lib.pretty is not maintained by its original creators anymore.

[njsmith]

There isn't much work to do to get the package working on Python 3.5. I had to remove Python 3.6 features to get the docs to build on ReadTheDocs.io (f-strings and use of enum.auto), so it probably runs on Python 3.5, but I haven't confirmed.

It does seem to run on 3.5, but there are small issues (datetime printer fails due to lack of fold attribute) and one larger one I noticed: the pretty_call function only gives you control over the order that kwargs are printed on 3.6, where kwargs preserve order. On 3.5, reprs using multiple kwargs will randomly change from run to run. This is an actual API issue, not an internal issue.

[tommikaikkonen]

the pretty_call function only gives you control over the order that kwargs are printed on 3.6, where kwargs preserve order. On 3.5, reprs using multiple kwargs will randomly change from run to run. This is an actual API issue, not an internal issue.

Good point. Python 3.5 users would have to use a different version of pretty_call, e.g. pretty_call_alt (can't think of a better name right now) with signature

pretty_call_alt(ctx, sorted, args=([3, 6, 4], ), kwargs=[('reversed', True)])`

to support ordered keyword arguments. It's not as simple as pretty_call, but works well and is an easy transformation to make the standard library definitions in PrettyPrinter compatible with Python 3.5. pretty_call would issue a warning if called with a Python version below 3.6.

[ivanov]

I'm supportive of the idea of having prettyprinter alternatives, but I would rather keep it on the level of having an extension point that's available for alternatives to what IPython provides by default, and being very conservative about changing that default because of how many people out there rely on IPython. But I'm just one person - and I realize there's a tension between keeping existing (grumpy-like-me) users happy by not changing things they are used to, and giving other users new reasons to make their experiences happier.