nbconvert: Latex template refactor #4112

jdfreder · 2013-08-26T18:53:21Z

I'd like as much input from everyone as possible. It's a good time to make sure feature X won't be too hard to implement in the future.

Goals

Minimize future pull request for latex style changes
Separate working parts of the templates
Best effort to de-obfuscate the latex

Motivation

Design

Conditional inheritance is used select an output style. This may seem strange, but it is required to allow inheriting output templates to override media blocks and those blocks to be used by the output style. It also is a nice way of circumventing the need to require a cell style template per output template.

Features

4 cell styles
- python
- ipython
- black & white python
- black & white ipython
2 output templates
- report
- article

python cell style

ipython cell style

black & white python cell style

black & white ipython cell style

ellisonbg · 2013-08-30T23:38:06Z

Did some in person review:

Move pygments definitions to the cell styles that needs them.
See if we really need alltt and/or fancyvrb?
Create Jinja2 blocks for title, author, date and abstract
Use inch margins all around
Use same name for draw_prompt macro in different styles
Sphinx release/version should be removed
Strip down the Sphinx preprocessor

ellisonbg · 2013-08-30T23:39:07Z

Cell styles that we want to support:

Simple
Notebook
- If we can simplify by getting rid of the colored/bordered box, we might
  be able to get rid of the fancy mdframed stuff and its problems
Python prompt version
- B&W
- Pygment colored
IPython prompt version
- B&W
- Pygment colored
- How to handle Out[] prompts?

jdfreder · 2013-09-04T20:48:04Z

Updated description, rebased, and ready to test/review.

jakobgager · 2013-09-04T21:01:53Z

The preview images look awesome! I hope to find some time in the near future to test this!

jdfreder · 2013-09-04T23:20:11Z

Thanks @jakobgager 😄

jdfreder · 2013-09-04T23:20:50Z

From in person review with @ellisonbg :

jdfreder · 2013-09-04T23:31:21Z

Also post review, added a bibliography block.

jdfreder · 2013-09-05T02:35:12Z

I updated the screenshots in the description to match the altered output for the ipython & bw ipython styles

minrk · 2013-09-05T17:27:38Z

from

nbconvert --to latex notebooks/Part\ 5\ -\ Rich\ Display\ System --post pdf

I get

[4 <./Part 5 - Rich Display System_files/Part 5 - Rich Display System_28_0.jpeg
>] [5]

! Package amsmath Error: \begin{aligned} allowed only in math mode.

See the amsmath package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.632 \nabla
             \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathb...

? 
! Emergency stop.

minrk · 2013-09-05T17:29:25Z

IPython/nbconvert/preprocessors/latex.py

+        """
+        # Generate Pygments definitions for Latex 
+        resources["latex"] = {}
+        resources["latex"]["pygment_definitions"] = LatexFormatter().get_style_defs()


minrk · 2013-09-05T17:42:25Z

in general, when creating a bunch of files of a given class, prefix seems better than suffix (style_ipython rather than ipython_style)

jdfreder · 2013-09-05T17:50:07Z

prefix seems better than suffix

Good tip, I've uploaded the changes

jdfreder · 2013-09-05T17:55:50Z

! Package amsmath Error: \begin{aligned} allowed only in math mode.

Sounds like the funky logic I wrote to fix the $$ tag spacing on math (https://github.com/ipython/ipython/blob/master/IPython/nbconvert/filters/latex.py#L66) is failing, could you paste a small portion of the tex around that line (632) please?

jakobgager · 2013-09-05T21:27:07Z

Funny issue. The first time I run pdflatex, latex complains about a missing language definition, and missing tikz.
However, when I run again this error disappears, even after deletion of all related files?

jdfreder · 2013-09-05T21:30:12Z

even after deletion of all related files?

Really? That is very very strange, haven't heard of anything like that so far. The new pdf post processor logic (that @ellisonbg wrote, which was just merged into master a couple days ago and has been rebased into this PR) should clean ALL of the temporary files between runs for you.

jdfreder · 2013-09-05T21:31:25Z

It should also automatically run pdflatex 3 times... I think it may abort if it encounters a critical failure, which may be a problem.. Maybe it should only abort on pdflatex failures after the final execution of pdflatex?

jakobgager · 2013-09-05T21:33:40Z

The following notebook converted fine with ipython 1.0.0 but results in an latex error using your branch:
http://nbviewer.ipython.org/c295019818fc09b669a4
Issue is due to In[4]

jdfreder · 2013-09-05T21:35:01Z

Thanks @jakobgager I'll take a look right now

jakobgager · 2013-09-05T21:39:36Z

Regarding the funny issue, it is quite tricky to reproduce the error. However I managed again (rename + switching from-and-to virtualenv) the error was

[NbConvertApp] Converting notebook Dir_List.ipynb to latex
[NbConvertApp] Support files will be in Dir_List_files/
[NbConvertApp] Loaded template latex_article.tplx
[NbConvertApp] Writing 6493 bytes to Dir_List.tex
(stream_Test)jakob@jakob-ThinkPad:~/Projekte/ipython$ pdflatex Dir_List
This is pdfTeX, Version 3.1415926-2.4-1.40.13 (TeX Live 2012/Debian)
 restricted \write18 enabled.
entering extended mode
(./Dir_List.tex
LaTeX2e <2011/06/27>
Babel <v3.8m> and hyphenation patterns for english, dumylang, nohyphenation, lo
aded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
...
   I removed some lines here
...
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty))

Package hyperref Message: Driver (autodetected): hpdftex.

(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty))
(./Dir_List.aux

! Package babel Error: You haven't loaded the option english yet.

See the babel package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.19 \select@language{english}

? 
! Undefined control sequence.
l.23 \pgfsyspdfmark
                    {pgfid1}{7815060}{36203544}
?

IIRC there was something similar with older versions, so it might not related to this at all.
Will investigate further.

jakobgager · 2013-09-05T21:40:35Z

Btw. the I used the default calls for both issues nbconvert --to latex foo.ipynb

jdfreder · 2013-09-05T21:42:30Z

l.23 \pgfsyspdfmark {pgfid1}{7815060}{36203544}

I used to get this error when I used the old simple cell style, left some of the temp files in the directory, and then tried to use the notebook cell style. The problem was that the temporary files didn't have the dependencies necessary to run the more complex tex. Still all the temporary files should be removed with the changes to the pdf post processor.... Not sure why you are seeing this

jdfreder · 2013-09-05T21:57:21Z

@jakobgager there was a missing escape_latex in the base, I just added it, please give a second whirl when you get a chance

jdfreder · 2013-09-05T21:57:56Z

It fixes the problem with your notebook on my machine.

jakobgager · 2013-09-06T21:04:23Z

Works also on my machine. I just wonder if the Output of In4 should get some linebreaks?

jdfreder · 2013-09-06T21:09:40Z

It probably should, if it's not too complicated, #3959

jdfreder · 2013-09-06T22:57:44Z

IPython/nbconvert/templates/latex/latex_base.tplx

+    ((* elif cell.level == 6 -*))
+        ((* block h6 -*))\\*\textit((* endblock h6 -*))
+    ((*- endif -*))
+    {((( cell.source | citation2latex | escape_latex )))}


TODO: Remove line breaks

deleted: latex/abstract.tplx deleted: latex/latex_sphinx_article.tplx deleted: latex/latex_sphinx_report.tplx deleted: latex/style_notebook.tplx deleted: latex/style_simple.tplx

Shouldn't have been checked in.

jdfreder · 2013-09-23T18:03:33Z

@ellisonbg and @minrk , changes made after our video conversation on Friday and rebased against master.

ellisonbg · 2013-09-23T21:13:40Z

IPython/nbconvert/preprocessors/sphinx.py

@@ -15,26 +15,9 @@
 #-----------------------------------------------------------------------------



We should move this file to nbconvert-examples with the rest of the sphinx template stuff.

ellisonbg · 2013-09-23T21:17:01Z

Overall this looks good, I had a few small comments inline. Lets make sure that the removed cell styles and sphinx stuff gets move over to nbconvert-examples.

ellisonbg · 2013-09-23T23:58:43Z

OK Travis is passing and everything looks good, merging.

nbconvert: Latex template refactor

ellisonbg mentioned this pull request Aug 27, 2013

Python cell style for latex output in nbconvert #4034

Closed

jdfreder mentioned this pull request Aug 28, 2013

fix vertical alignment issues, take 2 #4041

Closed

minrk reviewed Sep 5, 2013
View reviewed changes

jdfreder mentioned this pull request Sep 6, 2013

Nbconvert [windows]: Inconsistent line endings in markdown cells exported to latex #4122

Closed

jdfreder reviewed Sep 6, 2013
View reviewed changes

jdfreder mentioned this pull request Sep 6, 2013

add missing data_javascript #4178

Merged

jdfreder added 16 commits September 23, 2013 17:57

s/book/report in exporter and exporter tests

e69c588

Added citation support to latex templates

314de84

s/parse_citations/citations2latex

dd60010

style prefix instead of suffix

79db46d

Updated cell style references

cc392eb

s/pygment/pygments

c74425a

Fixed missing latex escape

de5e7c1

Removed invalid comment

d7d6a3f

Application of PR ipython#4181

5524115

Removed escape_latex on headers, was ruining citations

7e0d718

Added mathjax compatability definitions.

6aec763

Added markdown support to headers

5cd6fef

Typo fix

1a8b383

Post review with Min, Fernando, and Brian. Simplification

930d72c

deleted: latex/abstract.tplx deleted: latex/latex_sphinx_article.tplx deleted: latex/latex_sphinx_report.tplx deleted: latex/style_notebook.tplx deleted: latex/style_simple.tplx

deleted: latex/latex_article_simple.tplx

b5f8813

Shouldn't have been checked in.

Re-added addprompts() filter (Rebase)

903e87f

ellisonbg reviewed Sep 23, 2013
View reviewed changes

jdfreder added 2 commits September 23, 2013 21:35

Review changes

0e3e63b

Whoops! Removed sphinx references

5bd2f2e

ellisonbg added a commit that referenced this pull request Sep 23, 2013

Merge pull request #4112 from jdfreder/texrefact

04d60b8

nbconvert: Latex template refactor

ellisonbg merged commit 04d60b8 into ipython:master Sep 23, 2013

jdfreder mentioned this pull request Nov 14, 2013

Nbconvert: LaTeX book template not found #4537

Closed

jdfreder deleted the texrefact branch November 14, 2013 17:27

jakobgager mentioned this pull request Mar 11, 2014

Broken Layout with ipython nbconvert and BasicTex #5322

Closed

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#4112 from jdfreder/texrefact

6d3ad73

nbconvert: Latex template refactor

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nbconvert: Latex template refactor #4112

nbconvert: Latex template refactor #4112

jdfreder commented Aug 26, 2013

ellisonbg commented Aug 30, 2013

ellisonbg commented Aug 30, 2013

jdfreder commented Sep 4, 2013

jakobgager commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 5, 2013

minrk commented Sep 5, 2013

minrk Sep 5, 2013

minrk commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 6, 2013

jdfreder commented Sep 6, 2013

jdfreder Sep 6, 2013

jdfreder commented Sep 23, 2013

ellisonbg Sep 23, 2013

ellisonbg commented Sep 23, 2013

ellisonbg commented Sep 23, 2013

		@@ -15,26 +15,9 @@
		#-----------------------------------------------------------------------------

nbconvert: Latex template refactor #4112

nbconvert: Latex template refactor #4112

Conversation

jdfreder commented Aug 26, 2013

Goals

Motivation

Design

Features

python cell style

ipython cell style

black & white python cell style

black & white ipython cell style

ellisonbg commented Aug 30, 2013

ellisonbg commented Aug 30, 2013

jdfreder commented Sep 4, 2013

jakobgager commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 4, 2013

jdfreder commented Sep 5, 2013

minrk commented Sep 5, 2013

minrk Sep 5, 2013

Choose a reason for hiding this comment

minrk commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jakobgager commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jdfreder commented Sep 5, 2013

jakobgager commented Sep 6, 2013

jdfreder commented Sep 6, 2013

jdfreder Sep 6, 2013

Choose a reason for hiding this comment

jdfreder commented Sep 23, 2013

ellisonbg Sep 23, 2013

Choose a reason for hiding this comment

ellisonbg commented Sep 23, 2013

ellisonbg commented Sep 23, 2013