Adding PDFFormatter and kernel side handling of PDF display data #4920

ellisonbg · 2014-01-29T05:23:08Z

This is a continuation of #4130 to add PDF handling.

ellisonbg · 2014-01-29T07:16:15Z

OK the basics are now working, along with a refactored InlineBackend. But @takluyver and @minrk there is a problem with the %config magic that prevents the following from working:

%config InlineBackend.figure_formats = {'png','pdf'}

The problem is that the IPython input machinery sees the {} syntax as string fomatting and removes it leading to the expression 'png','pdf' which is a tuple. We have two options here:

Just make the figure_formats a List trait rather than Set.
Dig in a try to find out why the {} is getting removed.

ellisonbg · 2014-01-29T07:42:10Z

OK I have a simple OutputArea.append_pdf working. Here are screenshots of what it looks like:

And with just PDF:

ellisonbg · 2014-01-29T07:42:41Z

Looks like we have another bug: display(gcf()) doesn't work any longer.

ellisonbg · 2014-01-29T07:48:26Z

I have also tested this with nbconvert and it uses the embedded PDF files for latex/PDF output. Yeah!

Carreau · 2014-01-29T07:51:41Z

Note that this would partially conflict with one PR of mine. Also I think at some point we were wondering if we shoudln't have a @interpolate(False|True) for magics to expand {} or not.

Now go to sleep Brian :-)

jakobgager · 2014-01-29T08:35:50Z

Great!
If available, one could use Wand to create a thumbnail of the pdf to be displayed instead of, or in addition to the link, e.g. see http://stackoverflow.com/q/19470099/2870069

minrk · 2014-01-29T18:52:30Z

Disabling var_expand on several magics that don't actually want the extra magic would solve a lot of issues, but it's probably a task for another time.

In general, I'm not convinced that runtime config should really be the recommended approach for anything we expect people to actually change, since many configurables only make sense on first load, and can have weird, incomplete effects when changed at runtime. I would suggest making the recommended approach directly calling select_figure_format, and making that function available in an official public API.

minrk · 2014-01-30T19:34:15Z

IPython/kernel/zmq/pylab/config.py

@@ -79,6 +79,12 @@ def _figure_format_changed(self, name, old, new):
        else:
            select_figure_format(self.shell, new)

+    figure_format = Unicode()


This should still be configurable with a deprecation note

ellisonbg · 2014-01-30T20:59:46Z

The decision from the dev meeting is to:

Not solve the bug with how IPython magics handle {}.
Keep the config stuff as is.
Offer a new public function to setting the figure format.

Commits on the way...

ellisonbg · 2014-01-30T21:19:22Z

I just pushed a commit that adds this API:

IPython.display.select_matplotlib_formats(formats, quality=90)

Alternatives:

IPython.display.select_matplotlib_figure_formats(formats, quality=90)
IPython.display.set_mpl_figure_formats(formats, quality=90)

Also, much nicer to call most of the time is the *formats, **kwargs variation:

IPython.display.select_matplotlib_formats(*formats, **kwargs)

Thoughts?

ellisonbg · 2014-02-06T19:37:08Z

We decided set_matplotlib_formats(*formats, **kwargs) with a docstring telling people about quality=90.

ellisonbg · 2014-02-06T22:25:17Z

I have made the API changes discussed. For consistency I have also added a set_matplotlib_close function to IPython.display for interactive usage. I have also updated the docstrings of the various places that refer to this logic. I did rebase so we should wait for Travis to pass on this one.

fperez · 2014-02-06T23:58:45Z

IPython/core/display.py

+
+        In [1]: set_matplotlib_formats('png', 'jpeg', quality=90)
+
+    To set this in your config files use the following::


For equivalence with the example above, this should also indicate how to set the quality parameter in the config file.

fperez · 2014-02-07T00:02:20Z

Looks great. I made some small comments about docs, and it's also missing an entry in the what's new/api changes documentation sources indicating the new/changed API calls.

Once these small fixes are in, this looks good to merge to me.

ellisonbg · 2014-02-08T03:52:21Z

Review comments addressed, ready for merge.

Adding PDFFormatter and kernel side handling of PDF display data

westurner · 2014-02-15T18:41:51Z

@ellisonbg Two questions:

Should this be listed as a choice with ipython nbconvert --help
And/or: https://github.com/ipython/ipython/blob/master/examples/widgets/Export%20As%20(nbconvert).ipynb ?
How does this change http://ipython.org/ipython-doc/dev/interactive/nbconvert.html ?

westurner · 2014-02-15T18:48:15Z

.3. Will there be a 'Download As' > 'PDF (.pdf)`?

Trying to respond to this: http://www.reddit.com/r/IPython/comments/1xzjrx/ipython_founder_details_road_map_for_interactive/cfg1ex2

http://ipython.org/ipython-doc/rel-1.2.0/interactive/nbconvert.html

ipython nbconvert --to latex --template <article|book|basic> --post pdf notebook.ipynb

http://ipython.org/ipython-doc/dev/interactive/nbconvert.html

ipython nbconvert --to pdf --template <article|book|basic> notebook.ipynb

?

minrk · 2014-02-15T19:29:56Z

Will there be a 'Download As' > 'PDF (.pdf)`?

Yes, but not immediately. We plan to replace the --post PDF with a PDFExporter, which does basically the same thing. This is a prerequisite for download-as-pdf.

minrk · 2014-02-15T19:32:08Z

Should this be listed as a choice with ipython nbconvert --help

This affects execution, it does not affect the choices in nbconvert.

How does this change http://ipython.org/ipython-doc/dev/interactive/nbconvert.html

Not at all.

westurner · 2014-02-15T19:37:48Z

Cool; thanks. So will that require all 1G of texlive-full or all
1.2G of MacTeX?

rst2pdf/xhtml2pdf (reportlab) may also work.

On 2/15/14, Min RK notifications@github.com wrote:

Will there be a 'Download As' > 'PDF (.pdf)`?

Yes, but not immediately. We plan to replace the --post PDF with a
PDFExporter, which does basically the same thing. This is a prerequisite for
download-as-pdf.

Reply to this email directly or view it on GitHub:
#4920 (comment)

Wes Turner

minrk · 2014-02-15T19:43:16Z

Cool; thanks. So will that require all 1G of texlive-full or all 1.2G of MacTeX?

You can get by with basictex and then installing the missing packages with tlmgr, but I don't recall what they are off the top of my head.

ellisonbg · 2014-02-15T21:11:16Z

I should mention that with this change, if you have PDF output for a cell
it will automatically be used when you convert to LaTeX/PDF. This means no
more bitmapped matplotlib figures in nbconvert PDFs!

On Sat, Feb 15, 2014 at 11:43 AM, Min RK notifications@github.com wrote:

Cool; thanks. So will that require all 1G of texlive-full or all 1.2G of
MacTeX?

You can get by with basictex and then installing the missing packages with
tlmgr, but I don't recall what they are off the top of my head.

Reply to this email directly or view it on GitHubhttps://github.com//pull/4920#issuecomment-35165687
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

minrk · 2014-02-15T21:13:53Z

Oh, I didn't see that you added a application/pdf->pdf transform in the javascript. That should probably not have been added. I'll make a PR to remove that.

ellisonbg · 2014-02-15T21:40:29Z

I thought this was a originally a "not yet" thing. But once I got working,
I decided to make everything just work like other output formats. You can
view PDF output in the notebook UI and everything just works in nbconvert.
What will be the effect of that removal on these capabilities? What is the
plan to get the capabilities back?

On Sat, Feb 15, 2014 at 1:13 PM, Min RK notifications@github.com wrote:

Oh, I didn't see that you added a pdf->application/pdf transform in the
javascript. That should actually not have been added. I'll make a PR to
remove that.

Reply to this email directly or view it on GitHubhttps://github.com//pull/4920#issuecomment-35168009
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

minrk · 2014-02-16T02:55:01Z

The "not yet" part is fixing the renames that we already do. That doesn't mean that any new formats should get their own rename, knowing that it will just be removed later. We can already have arbitrary mimetype output in the notebook document, so I don't think there's a reason to add a rename for pdf.

What will be the effect of that removal of these capabilities? What is the
plan to get the capabilities back?

No capabilities are affected. The only thing I removed in #5132 is renaming 'application/pdf' to 'pdf', which did not add anything.

ellisonbg · 2014-02-17T20:17:11Z

OK, thanks for the clarification, sounds good.

On Sat, Feb 15, 2014 at 6:55 PM, Min RK notifications@github.com wrote:

The "not yet" part is fixing the renames that we already do. That doesn't
mean that any new formats should get their own rename, knowing that it
will just be removed later. We can already have arbitrary mimetype output
in the notebook document, so I don't think there's a reason to add a rename
for pdf.

What will be the effect of that removal of these capabilities? What is the
plan to get the capabilities back?

No capabilities are affected. The only thing I removed in #5132 https://github.com/ipython/ipython/pull/5132is renaming 'application/pdf' to 'pdf', which did not add anything.

Reply to this email directly or view it on GitHubhttps://github.com//pull/4920#issuecomment-35175765
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

Adding PDFFormatter and kernel side handling of PDF display data

ellisonbg mentioned this pull request Jan 29, 2014

Add PDF formatter and handling #4921

Closed

5 tasks

minrk reviewed Jan 30, 2014
View reviewed changes

ellisonbg added this to the 2.0 milestone Feb 5, 2014

ellisonbg self-assigned this Feb 5, 2014

ellisonbg added 7 commits February 6, 2014 14:11

Adding PDFFormatter and kernel side handling of PDF display data.

f86e628

Adding support for multiple figure formats in InlineBackend.

9add814

Making figure_format config=True.

138e66d

Adding top level IPython.display.select_matplotlib_formats.

546225e

Docstring improvement.

db5f40f

Refining the matplotlib APIs in IPython.display.

756f4ef

Adding basic append_pdf in OutputArea.

67f7adb

fperez reviewed Feb 6, 2014
View reviewed changes

ellisonbg added 2 commits February 7, 2014 19:46

Adding line about setting quality in config.

c59a5a2

Fixing docstring.

1008630

ellisonbg added a commit that referenced this pull request Feb 8, 2014

Merge pull request #4920 from ellisonbg/pdf-formatter

e68e13c

Adding PDFFormatter and kernel side handling of PDF display data

ellisonbg merged commit e68e13c into ipython:master Feb 8, 2014

ellisonbg deleted the pdf-formatter branch February 8, 2014 19:49

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#4920 from ellisonbg/pdf-formatter

713e2d8

Adding PDFFormatter and kernel side handling of PDF display data

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding PDFFormatter and kernel side handling of PDF display data #4920

Adding PDFFormatter and kernel side handling of PDF display data #4920

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

Carreau commented Jan 29, 2014

jakobgager commented Jan 29, 2014

minrk commented Jan 29, 2014

minrk Jan 30, 2014

ellisonbg commented Jan 30, 2014

ellisonbg commented Jan 30, 2014

ellisonbg commented Feb 6, 2014

ellisonbg commented Feb 6, 2014

fperez Feb 6, 2014

ellisonbg Feb 8, 2014

fperez commented Feb 7, 2014

ellisonbg commented Feb 8, 2014

westurner commented Feb 15, 2014

westurner commented Feb 15, 2014

minrk commented Feb 15, 2014

minrk commented Feb 15, 2014

westurner commented Feb 15, 2014

minrk commented Feb 15, 2014

ellisonbg commented Feb 15, 2014

minrk commented Feb 15, 2014

ellisonbg commented Feb 15, 2014

minrk commented Feb 16, 2014

ellisonbg commented Feb 17, 2014


		In [1]: set_matplotlib_formats('png', 'jpeg', quality=90)

		To set this in your config files use the following::

Adding PDFFormatter and kernel side handling of PDF display data #4920

Adding PDFFormatter and kernel side handling of PDF display data #4920

Conversation

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

ellisonbg commented Jan 29, 2014

Carreau commented Jan 29, 2014

jakobgager commented Jan 29, 2014

minrk commented Jan 29, 2014

minrk Jan 30, 2014

Choose a reason for hiding this comment

ellisonbg commented Jan 30, 2014

ellisonbg commented Jan 30, 2014

ellisonbg commented Feb 6, 2014

ellisonbg commented Feb 6, 2014

fperez Feb 6, 2014

Choose a reason for hiding this comment

ellisonbg Feb 8, 2014

Choose a reason for hiding this comment

fperez commented Feb 7, 2014

ellisonbg commented Feb 8, 2014

westurner commented Feb 15, 2014

westurner commented Feb 15, 2014

minrk commented Feb 15, 2014

minrk commented Feb 15, 2014

westurner commented Feb 15, 2014

minrk commented Feb 15, 2014

ellisonbg commented Feb 15, 2014

minrk commented Feb 15, 2014

ellisonbg commented Feb 15, 2014

minrk commented Feb 16, 2014

ellisonbg commented Feb 17, 2014