nbconvert: Export flavors & PDF writer (ipy dev meeting) #3784

jdfreder · 2013-07-25T22:28:31Z

Lightweight subclasses reduced in quantity
Added post processors
--format param is now --to

--post PDF flag added which creates and uses the new (introduced in this PR) PDFPostProcessor

See the following excerpt from help:

You can specify the export format with `--to`.
Options include {0}

> ipython nbconvert --to latex mynotebook.ipnynb

Both HTML and LaTeX support multiple output templates. LaTeX includes
'basic', 'book', and 'article'.  HTML includes 'basic' and 'full'.  You 
can specify the flavor of the format used.

> ipython nbconvert --to html --template reveal mynotebook.ipnynb

You can also pipe the output to stdout, rather than a file

> ipython nbconvert mynotebook.ipynb --stdout

A post-processor can be used to compile a PDF

> ipython nbconvert mynotebook.ipynb --to latex --post PDF

See requirements listed in dev meeting hackpad:
https://hackpad.com/IPython-Summer-2013-Development-Meeting-D1UR23usGnA

jhamrick · 2013-07-25T22:37:05Z

There was some discussion about referring to reveal as just "slides" or "presentation", e.g.:

> ipython nbconvert --to html --flavor slides mynotebook.ipnynb

It's essentially the same idea as not mentioning sphinx with the LaTeX book/article flavor. Is that something we still want to do?

minrk · 2013-07-25T22:42:12Z

I think it was also considered that reveal would be --to slides --flavor reveal. It remains unclear to me what is best.

jdfreder · 2013-07-25T22:43:29Z

@jhamrick I talked to @ellisonbg about that a couple minutes ago, he prefered --to html --flavor reveal.

Personally I prefer --to html --flavor slides

I decided against --to slides or --to slides --flavor reveal because slides really isn't a data format, where all the other choices are.

jdfreder · 2013-07-25T22:43:59Z

We should talk about it again... I'll bring it up when I show this stuff working

jdfreder · 2013-07-25T22:45:05Z

minutes = hours*

damianavila · 2013-07-25T22:45:26Z

I agree with @minrk... I prefer --to slides --flavor reveal because if the ipython team is interesting I can do:

--to slides --flavor impress
or
--to slides --flavor deck
or
--to slides --flavor flowtime

all other slideshows libraries I want to add in the future ;-)

damianavila · 2013-07-25T22:48:36Z

if not inside IPython core, if I release this flavors as extension, I would be great to have a sintaxis as --to slides --flavor whatever ;-)

jdfreder · 2013-07-25T22:50:32Z

I guess data format isn't what I should have been thinking about, since the whole reason for the subclasses is the ability to configure based on that particular format... I'll change it to slides, one minute...

jhamrick · 2013-07-25T22:54:26Z

@damianavila I guess the question is, does IPython want to have just one type of HTML slides/presentation that it natively supports, or many? There is also going to be the option to use --template to specify a specific jinja template (e.g. --template impress.tpl). I would lean more towards having just one default type of slides in the IPython core, and if you want to use something beyond that, specify a specific template. My guess would be that someone who just wants to create a presentation and doesn't really care what the backend is would be confused by the various options. I suppose if we had --to slides then that could just have a reasonable default.

@jdfreder You mean change it to --to slides? If support for beamer slides ever got added, would that then go under --to slides --flavor beamer or --to latex --flavor slides?

jdfreder · 2013-07-25T22:56:05Z

yes, and the first of the two

jhamrick · 2013-07-25T23:00:17Z

Hmm, ok. It could be a bit confusing because beamer is LaTeX (and presumably --to slides --flavor beamer would output a .tex file -- as would all the --to latex conversions). Maybe there isn't really a good solution though; it's a tradeoff between how obvious the flags are and how many options/flavors can be supported.

jakobgager · 2013-07-25T23:01:33Z

I also agree with @minrk. In the end the outcome should be what matters and a presentation is something complete different than a webpage! In this context LaTeX Beamer would be an extreme case of an presentation, even though a tex file.

@damianavila deck looks awesome!

damianavila · 2013-07-25T23:03:15Z

@jhamrick probably IPython will support one type of HTML slides/presentations at the core (@minrk @fperez @ellisonbg @Carreau thoughs?) ... but the proposed syntax (--to slides --flavor whatever) will help me (and others) to bring another kind of HTML slides as extensions.
About custom templating... there will be the possibility to add you own template, in that case something like --to slides --flavor whatever --template=custom_tpl will be easy to understand and gives you the possibility of multiple outputs for the same "flavor"...

damianavila · 2013-07-25T23:05:47Z

@jakobgager yes, all this js libraries are very powerful, just I need some more time to make them available... they are in my todo list (a very large one... hehe).

jdfreder · 2013-07-25T23:10:21Z

Hey all, I change it so --to slides works and defaults to reveal flavor. In the future the default flavor may change, but explicitly exporting to --to slides --flavor reveal will always export to reveal.

jhamrick · 2013-07-25T23:13:50Z

@jakobgager Yeah, fair enough.

@damianavila I think the idea with templates was that they would be instead of a flavor, rather than in addition to. The analogy Fernando used I believe was that you can order e.g. a hawaiian pizza (flavor), but if you want a hawaiian pizza with spinach you need to just give the restaurant a new recipe of ham+pineapple+spinach (template).

jdfreder · 2013-07-25T23:15:58Z

@jhamrick exactly, that said, --template still works... I think I forgot to mention that. --template overrides --flavor

damianavila · 2013-07-25T23:42:07Z

@jdfreder thx for the clarification...

@jhamrick you are right, I was thinking in other sort of structure without reading the code ;-)

jdfreder · 2013-07-26T01:15:00Z

--template now includes the functionality of --flavor. --flavor was removed.

ipython nbconvert --to latex --pdf
ipython nbconvert --to latex --template book --pdf
ipython nbconvert --to latex --template latex_book --pdf
ipython nbconvert --to latex --template latex_book.tplx --pdf

should all work...

jdfreder · 2013-07-26T01:30:27Z

Don't merge yet

damianavila · 2013-07-26T12:21:54Z

One thing it is not clear to me... what about the location of the custom template file? I mean, if I wrote a custom template and I want to use it doing ipython nbconvert --to latex --template my_template, where I have to locate my_template? Inside the templates folder of the IPython installation? or how I pass this location to the FileSystemLoader in an easy way?

Carreau · 2013-07-26T15:44:17Z

where I have to locate my_template? Inside the templates folder of the IPython installation?

This will work. There is also IIRC the possibility to add custom search locations.

or how I pass this location to the FileSystemLoader in an easy way?

the worth that can happend is you have to instanciate a loader in your config file:

from jinja import FilesystemLoader

c.IDontRemember.extra_loader = [FilesystemLoader('my/path')]

Advantages are you don't even need to use a template file, you could use a dict loader, or a mondodb loader...

it is slightly more complicated that just giving a path, but nbconvert is still a tech preview, and I thing we are already trying to over-engenier it.

damianavila · 2013-07-26T16:20:32Z

There is also IIRC the possibility to add custom search locations.

Overwriting this one? https://github.com/jdfreder/ipython/blob/98ef91d1c49a185ff508bb31ba014465b1249df3/IPython/nbconvert/exporters/exporter.py#L111

instanciate a loader in your config file:

Oh, I missed that point, you can add extra_loaders... great!

it is slightly more complicated that just giving a path

Yeah, maybe in the future we can provide the UI at the command line add your custom template.

Carreau · 2013-07-26T16:31:13Z

Overwriting this one?

No, because it does not have config=True, but the one 2 lines above yes, and it will take precedence.

Yeah, maybe in the future we can provide the UI at the command line add your custom template.

You mean template path because you can specify template from CL. actually you can even write a myconfig.py and just do ipython nbconvert --config=myconfig.py and "it works".

damianavila · 2013-07-26T17:16:25Z

You mean template path

Yes, I was talking about the path, sorry to not be clear enough.

damianavila · 2013-07-26T17:17:57Z

No, because it does not have config=True, but the one 2 lines above yes, and it will take precedence.

Great... thanks!

jdfreder · 2013-07-26T17:19:08Z

@damianavila You should be able to put it in your cwd once my other PR gets merged ( #3752 )... It seems to be not moving anywhere though

Also, template_file (alias --template), behaves both as the flavor (a shortcut for template name) and the template . See the following snipped from Exporter.py, lines 206-209

try_names = [self.template_file + self.template_extension,
    self.template_file,
    module_name + '_' + self.template_file + self.template_extension,
    module_name + self.template_extension]

damianavila · 2013-07-26T17:31:08Z

@jdfreder Thanks for the info about cwd... I did not know about that PR... I have to look deeper before asking ;-)

jdfreder · 2013-07-26T17:57:13Z

Just rebased, adding @ivanov 's switch to the new tests...
Ready for code review

damianavila · 2013-07-26T22:47:09Z

I am reading this in detail... I will report any doubts soon...

minrk · 2013-07-26T22:47:52Z

now needs rebase

jdfreder · 2013-07-26T22:50:52Z

Just rebased

damianavila · 2013-07-26T22:56:43Z

IPython/nbconvert/exporters/slides.py

@@ -1,6 +1,7 @@
 """
-Reveal slide show exporter.
+Exporter that exports Basic HTML.


Exporter that exports HTML slides.

minrk · 2013-07-26T23:25:40Z

@ellisonbg and @fperez didn't like that, I'm indifferent. The idea was that --pdf would encourage someone to try --html etc.

Good enough for me.

damianavila · 2013-07-26T23:40:21Z

IPython/nbconvert/nbconvertapp.py

@@ -83,14 +100,24 @@ def _classes_default(self):

        which will convert mynotebook.ipynb to the default format (probably HTML).

-        You can specify the export format with `--format`.
+        You can specify the export format with `--to`.
        Options include {0}


Options include ?

This gets formatted below

damianavila · 2013-07-27T00:59:03Z

IPython/nbconvert/exporters/exporter.py

+        try_names = [self.template_file + self.template_extension,
+                     self.template_file,
+                     module_name + '_' + self.template_file + self.template_extension,
+                     module_name + self.template_extension]


Can you tell me a use case for this last one? maybe I am tired ;-) but I can't figure it out a use case... thanks

Yeah, ipython nbconvert --to rst notebook1.ipynb

Or any other format without flavors

This is used for the flavor. When you have a default templates like markdown that don't have flavor like 'markdown_foo.tpl'

OK, thanks to both of you... I was thinking only in "flavored" things... got it!

damianavila · 2013-07-27T01:37:36Z

OK, thanks to all for the clarifications... 👍 to merging ;-)

damianavila · 2013-07-27T04:12:22Z

IPython/nbconvert/nbconvertapp.py

@@ -112,6 +140,7 @@ def _classes_default(self):
                                    help="""Writer class used to write the 
                                    results of the conversion""")
    writer_aliases = {'FilesWriter': 'IPython.nbconvert.writers.files.FilesWriter',
+                      'PDFWriter': 'IPython.nbconvert.writers.pdf.PDFWriter',


This have to be erased because it is no longer more a writer ;-)

Ahh Thank you so much @damianavila for pointing this out! This may be why the tests failed

You are welcome... now in the morning I'm more attentive ;-)

nbconvert: Export flavors & PDF writer (ipy dev meeting)

Carreau · 2013-07-27T15:46:08Z

Yeaaaaah

damianavila · 2013-07-27T15:50:00Z

Great!

jgoppert · 2014-04-10T00:37:33Z

+1 for beamer slides flavor output. The recent changes to the template nbconvert backend allowed me to hack my own beamer support. It isn't perfect, but works well enough for me.

I'm using the reveal slide metadata info from IPython notebook to guide the latex creation. Also, I'm using the warsaw default theme that makes the section automatically appear at the top of the page so you don't have to worry about titling slides.

This template below/ embedded in the bash script, should serve as a decent latex beamer flavor starting point.

#!/bin/bash
curdir=$(pwd)
if [ $# != 4 ]
then
    echo "usage: $0 author date title notebook"
    exit -1
else
    author=$1
    date=$2
    title=$3
    notebook=$4
fi
rm -rf tmp
mkdir -p tmp
cd tmp
echo """
((*- extends 'base.tplx' -*))

((* block docclass *))
\documentclass{beamer}
\usetheme{Warsaw}
((* endblock docclass *))

((* block margins *))
((* endblock margins *))

((* block title *))
    \title{$title}
((* endblock title *))

((* block date *))
    \date{$date}
((* endblock date *))

((* block author *))
    \author{$author}
((* endblock author *))

((* block packages *))
    ((( super() )))
((* endblock packages *))

% Use reveal.js tags to handle skipping/ new slides etc
((* block any_cell *))
    ((* if cell.metadata.get('slideshow',{}).get('slide_type','-').lower() in ['skip'] *))
    ((* elif cell.metadata.get('slideshow',{}).get('slide_type','-').lower() in ['slide'] *))
        ((* if cell.level == 0 -*))
            \frame{\section
            \frame{
            ((( super() )))
            }
        ((* elif cell.level == 1 -*))
            \section{ ((( cell.source ))) }
            \frame{\sectionpage}
        ((* endif -*))
    ((* elif cell.metadata.get('slideshow',{}).get('slide_type','-').lower() in ['subslide'] *))
        \frame{
        ((( super() )))
        }
    ((* elif cell.metadata.get('slideshow',{}).get('slide_type','-').lower() in ['fragment'] *))
        ((( super() )))
    ((* else *))
        ((( super() )))
    ((* endif *)) 
((* endblock any_cell *))
""" > notebook.tplx
ipython nbconvert --quiet --to=latex \
    --template="notebook.tplx" \
    --post=PDF "$curdir/$notebook"
# vim:ts=4:sw=4:expandtab

nbconvert: Export flavors & PDF writer (ipy dev meeting)

jdfreder mentioned this pull request Jul 26, 2013

Added serving the output directory if html-based format are selected. #3780

Merged

jdfreder added 4 commits July 26, 2013 15:46

Remove executable comments from non-standalone files

cc54e59

Fix tests

d59fec6

Added warnings

d9ef1c7

Removed bash string

795cbfe

damianavila reviewed Jul 26, 2013
View reviewed changes

jdfreder added 2 commits July 26, 2013 16:14

Fix docstring

d95e89d

Make extension more descript

8912859

damianavila reviewed Jul 26, 2013
View reviewed changes

Update comment

22b9373

damianavila reviewed Jul 27, 2013
View reviewed changes

jdfreder added 2 commits July 27, 2013 08:27

Fixed tests (because of reveal extension change)

c1cdf94

Removed non-existant PDFWriter alias

e974c91

jdfreder added a commit that referenced this pull request Jul 27, 2013

Merge pull request #3784 from jdfreder/export_flavors

a9c0f49

nbconvert: Export flavors & PDF writer (ipy dev meeting)

jdfreder merged commit a9c0f49 into ipython:master Jul 27, 2013

jdfreder mentioned this pull request Jul 30, 2013

update nbconvert doc with new CLI #3831

Merged

jdfreder deleted the export_flavors branch March 10, 2014 18:42

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#3784 from jdfreder/export_flavors

d57f5d8

nbconvert: Export flavors & PDF writer (ipy dev meeting)

nbconvert: Export flavors & PDF writer (ipy dev meeting) #3784

nbconvert: Export flavors & PDF writer (ipy dev meeting) #3784

Conversation

jdfreder commented Jul 25, 2013

jhamrick commented Jul 25, 2013

minrk commented Jul 25, 2013

jdfreder commented Jul 25, 2013

jdfreder commented Jul 25, 2013

jdfreder commented Jul 25, 2013

damianavila commented Jul 25, 2013

damianavila commented Jul 25, 2013

jdfreder commented Jul 25, 2013

jhamrick commented Jul 25, 2013

jdfreder commented Jul 25, 2013

jhamrick commented Jul 25, 2013

jakobgager commented Jul 25, 2013

damianavila commented Jul 25, 2013

damianavila commented Jul 25, 2013

jdfreder commented Jul 25, 2013

jhamrick commented Jul 25, 2013

jdfreder commented Jul 25, 2013

damianavila commented Jul 25, 2013

jdfreder commented Jul 26, 2013

jdfreder commented Jul 26, 2013

damianavila commented Jul 26, 2013

Carreau commented Jul 26, 2013

damianavila commented Jul 26, 2013

Carreau commented Jul 26, 2013

damianavila commented Jul 26, 2013

damianavila commented Jul 26, 2013

jdfreder commented Jul 26, 2013

damianavila commented Jul 26, 2013

jdfreder commented Jul 26, 2013

damianavila commented Jul 26, 2013

minrk commented Jul 26, 2013

jdfreder commented Jul 26, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minrk commented Jul 26, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damianavila commented Jul 27, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Carreau commented Jul 27, 2013

damianavila commented Jul 27, 2013

jgoppert commented Apr 10, 2014