Download as pdf via LaTex not working

[cniedotus]

#143 at IRKernel

[takluyver]

Copying what I suspect is the relevant part of the Latex output:

[1{/usr/local/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
[2]

! LaTeX Error: Environment description* undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type H for immediate help.
...

l.423 \begin{description*}

?
! Emergency stop.
...

l.423 \begin{description*}

! ==> Fatal error occurred, no output PDF file produced!

[cniedotus]

Any idea about how I can fix it? The Download as pdf function works for Python 2 Kernel.

[takluyver]

Sorry, I don't really know about latex. Try to work out what bit of output it's choking on, that might help.

[alvinwt]

I tried generating the pdf using nbconvert and it failed at the compilation step. When I do pdflatex on the generated .tex file it works. A workaround might be using nbconvert at the commandline and compiling using pdflatex.

[dkincaid]

It would be really nice to get this fixed. Any estimate of when it might get fixed?

[takluyver]

I think it's waiting for someone who knows LaTeX to investigate what's going on. Volunteers welcome :-)

[dkincaid]

Well, I know Latex well, but don't know anything about how all this Jupyter/IRKernel/nbconvert connects together. I'll see if I can poke around a bit.

[takluyver]

You can generate the LaTeX by doing nbconvert by --to latex, and then run pdflatex on that. Although @alvinwt reports that that makes the bug disappear for him, so I'm not sure what's going on. The latex->pdf conversion code is here:

nbconvert/nbconvert/exporters/pdf.py

Line 127 in 0e6466d

with TemporaryWorkingDirectory() as td:

[dkincaid]

The problem is that for some reason the export is using "description*" as the list environment. As far as I know there is no such environment. At least not one in any of the common libraries. It should really be changed to "description" instead, but I cannot find where that typo is in the code. Can anyone help find it or at least point me to where it might be?

There is at least one other problem too. It's not escaping percent signs. It's using "%" when it should be using "%"

[alvinwt]

Hi @dkincaid, I tried looking for the description* tag and it's not in the nbconvert templates as well. Interestingly, the description* block matches the output of the IRKernel.

\begin{description*}
\item[X56403463.1727.4c23.a709.de5f168c4073] 50.0006258799053
\item[ca8afb2c.be3d.41e8.8dbc.19fe7ab6154b] 30.3254246898821
\item[CHC433] 21.169034458756
\item[X65e430e1.0c9a.4045.83a9.653b7cff811d] 14.2808853888554
\item[X6dcfc418.ab28.4365.95ea.c1ae254f2341] 11.8185201121792
\end{description*}

json from notebook

      "text/latex": [
       "\\begin{description*}\n",
       "\\item[X56403463.1727.4c23.a709.de5f168c4073] 50.0006258799053\n",
       "\\item[ca8afb2c.be3d.41e8.8dbc.19fe7ab6154b] 30.3254246898821\n",
       "\\item[CHC433] 21.169034458756\n",
       "\\item[X65e430e1.0c9a.4045.83a9.653b7cff811d] 14.2808853888554\n",
       "\\item[X6dcfc418.ab28.4365.95ea.c1ae254f2341] 11.8185201121792\n",
       "\\end{description*}\n"

Output from the notebook

Out[72]:
X56403463.1727.4c23.a709.de5f168c4073
50.0006258799053
ca8afb2c.be3d.41e8.8dbc.19fe7ab6154b
30.3254246898821
CHC433
21.169034458756
X65e430e1.0c9a.4045.83a9.653b7cff811d
14.2808853888554
X6dcfc418.ab28.4365.95ea.c1ae254f2341
11.8185201121792

Interestingly the description tag is in the .ipynb file and rendered in the "text/latex" section in the output. I'm not sure how the notebook handles this but I couldn't find the same section in my notebook with the python kernels.

[takluyver]

Aha! I hadn't thought of this coming from the kernel itself. It looks like the relevant code is here:

https://github.com/IRkernel/repr/blob/1f37cacab435d29da5e0679a4be75bd0899ab565/R/repr_vector.r#L122

And here's the commit that added it: IRkernel/repr@8f50f10

@flying-sheep, the description* environment seems to be causing problems in nbconvert. Any thoughts on what can be done?

[flying-sheep]

what can be done?

add a way to register stuff to add to the LaTeX preample of the template.

right now the default template has a rather random assortment of \usepackage{} lines that fits specifically the needs of IPython’s LaTeX output. that’s not flexible anough, and we can’t add everything everyone would ever want, so we need a way to specify what we want.

as documented in help(package = 'repr'), IRkernel/repr needs \usepackage[inline]{enumitem} and possibly \usepackage{minted}

[dkincaid]

I can confirm that adding \usepackage[inline]{enumitem} fixes the problem with the missing description* environment. However, I don't see that documented in help(package = 'repr') as was suggested.

Adding minted causes other issues, so I would recommend against including that.

Please don't forget about the issue with % characters not being escaped as well. That will cause additional compilation errors once the description error is resolved.

[flying-sheep]

oh sorry, i meant package?repr

Please don't forget about the issue with % characters not being escaped as well. That will cause additional compilation errors once the description error is resolved.

where does the % problem happen?

[dkincaid]

This piece of code:

quantile(raw_in_gb, seq(0.1,1,0.1))

produces this output in Jupyter notebook:

10% 0
20% 0
30% 0
40% 0.0000892743291503907
50% 0.0007269946
60% 0.00906304677999997
70% 0.0788954083100001
80% 0.56371782982
90% 2.9702177811
100% 394.2425975632

and this output in latex after nbconvert is run:

    \begin{description*}
\item[10%] 0
\item[20%] 0
\item[30%] 0
\item[40%] 0.0000892743291503907
\item[50%] 0.0007269946
\item[60%] 0.00906304677999997
\item[70%] 0.0788954083100001
\item[80%] 0.56371782982
\item[90%] 2.9702177811
\item[100%] 394.2425975632
\end{description*}

in Latex the % characters need to be escaped as \%

[flying-sheep]

hmm, right. tracked as IRkernel/repr#10

[matrs]

I just got this problem trying to convert a R notebook, so it's still here, any news?

LaTeX Error: Environment enumerate* undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type H for immediate help.
...
l.303 \begin{enumerate_}
?
! Emergency stop.
...
l.303 \begin{enumerate_}
! ==> Fatal error occurred, no output PDF file produced! >

[flying-sheep]

as said above, that’s due to

1. my choice to use the enumitem package
2. the default latex template hardcoding the packages that someone at some point decided IPython would use

you can put this in a file r.tplx:

((*- extends 'article.tplx' -*))

((* block packages *))    
    ((( super() )))

    \usepackage[inline]{enumitem}
    \usepackage{minted}
((* endblock packages *))

and then call nbconvert --to=pdf --template=r.tplx something.ipynb

[matrs]

Thanks, but as it was mentioned earlier, that only works without package {minted}, once i erased that line it worked.
Package ifplatform Warning: shell escape is disabled, so I can only detect \ifwindows. ) (/usr/share/texmf-dist/tex/latex/etoolbox/etoolbox.sty) (/usr/share/texmf-dist/tex/generic/xstring/xstring.sty (/usr/share/texmf-dist/tex/generic/xstring/xstring.tex)) (/usr/share/texmf-dist/tex/latex/lineno/lineno.sty)) ! Package minted Error: You must invoke LaTeX with the -shell-escape flag.

Also, there are many rendering errors in the PDF, errors that when I create a python nb instead of a R one , and use %%R cell magic and convert the nb to pdf don't happen.

For example
from and R nb to pdf you will see a number by every letter.

obs<-c("r","d","r","d","r","d","d","r","r","d","d","r","d","r","r","r","d","r","d","r","d","r","d","r","r","d","r","r","d","d","r","r","r","d","r","r","d","d","d","d","r","r","d","r","d","r","d","r","d","r")
obs

From a python nb with %%R the rendering is correct.

[flying-sheep]

not an error. that’s the vector indices 😄

if you want them to be displayed differently, read the enumitem docs and find out how to configure them to show no indices.

i think you can use \setlist[enumerate*]{...} or so

[matrs]

Of course are the indexes but in dont see them in R console, i dont see them in the the html from the R notebook, i dont see them in the pdf created by ipython using R magic so why i suppose to want them to be displayed in the pdf ? Not an error? displaying something that nobody sees is not an error?. It isnt a matter of preference, or if I want them to be displayed differently, no program is showing them except this nbconvert from R notebooks.
When you render a pdf from a notebook, is fair to expect in the pdf the same content that you're seeing in the html, just like the pdf from ipython nbconvert does. R does not show the indexes of every element, just one number per row and all programs mentioned show the same, all of them.

And finally, if nbconvert from a ipython notebook using R magic produces a pdf displaying what it is expected, without dealing with any kind of tex configuration, why nbconvert from and R nb does not?

[flying-sheep]

haha no reason to get worked up either.

it’s just the representation i chose, because there’s no easy way to get “display index of the index of the first item per line” in LaTeX.

you can configure my representation by calling \setlist with the right options (one reason why i chose enumitem instead of LaTeX enumerations or “numbers with spaces between them”). you just have to read the docs that i linked to 😺

all this is a byproduct of using fancy displaying instead of ugly “render R’s output as monospaced text” (what R magic does). Try displaying a matrix or data.frame and you’ll see why this is better!

please share the \setitem invocation that does what you want and i’ll put it in the docs 👍

[gidden]

Hi all, any movement on this issue?

I added the following r.tplx to ~/.local/share/jupyter/templates

((*- extends 'article.tplx' -*))

((* block packages *))
    ((( super() )))

    \usepackage[inline]{enumitem}
((* endblock packages *))

Then tried to convert my notebook

$ jupyter nbconvert --to pdf --template r.tplx hh_data.ipynb 

which resulted in the output here. The important part is below:

! Missing $ inserted.
<inserted text> 
                $
l.281 ...SCRIPTION & URL & ISO3 & PERIOD & SURVEY_
                                                  NAME\\
? 
! Emergency stop.
<inserted text> 
                $
l.281 ...SCRIPTION & URL & ISO3 & PERIOD & SURVEY_
                                                  NAME\\
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on notebook.log.

Any help is appreciated. Cheers!

[flying-sheep]

can you try with the newest IRkernel/repr from github? this is a latex escaping error which was hopefully addressed in a recent PR

devtools::install_github('IRkernel/repr')

[gidden]

Hi @flying-sheep, I did a devtools::install_github('IRkernel/repr', lib='~/.local/R/library', force=TRUE), still have the same error (both with file->download as and using the CLI).

[gidden]

Is it possible to confirm the R kernel version being used through a CLI or API in the notebook?

[flying-sheep]

sure, packageVersion('packagename').

about the error: that’s really weird. this line should take care of col/rownames, and those of values.

what object causes the breakage (contains SURVEY_NAME)? what does repr::repr_latex(the_object) output?

[gidden]

In [5]:

packageVersion('repr')

Out[5]:

[1] ‘0.4’

In [6]:

repr::repr_latex(dbReadTable(conn, 'SURVEY_DATA'))

Out[6]:
'
\begin{tabular}{r|lllllll}
  & SURVEY & COUNTRY & DESCRIPTION & URL & ISO3 & PERIOD & SURVEY\_NAME\\
\hline
1 & IND1 & India & NA & NA & IND & 2011-2012 & IND NSS 68 2011-2012\\
\end{tabular}
'

[flying-sheep]

well, it can’t be sthe same error still as. now repr outputs \_ instead of _.

that should be fine. you’ll have to recreate the cell outputs though, maybe you forgot that.

repr is only responsible to create the LaTeX code, but once (faulty) code is saved in the .ipynb file it can’t change it.

[gidden]

Perhaps this is a separate issue, then? I fully upgraded jupyter (from pip) and have tried to use both the GUI and CLI to convert, but still get failures. I tried converting to just latex and the table output is not the same as what repr::repr_latex provides (i.e., jupyter nbconvert does not appear to be using the latest IRkernel installed on my box).

[takluyver]

With the command you have above, nbconvert won't use IRkernel at all - it's working with the outputs saved in your notebook. You'll need to re-run the notebook after updating IRkernel to get the updated output.

There is a --execute flag for nbconvert which will make it re-run the code and regenerate the outputs.

[gidden]

Hi @takluyver, should have been more precise. I killed the notebook process after installing the updated kernel. Note that the repr::repr_latex output is actually copy-pasted from a notebook. The GUI download as feature still fails (500 internal server error nbconvert failed: PDF creating failed)

[gidden]

Ok, finally figured it out. I had to

1. update IRKernel
2. update jupyter
3. kill all active kernels and restart

Thanks for the assist!

[flying-sheep]

sorry to be nitpicky, but i think what you really had to do is

1. unload and reload repr in the kernel (probably easiest done by killing the kernel and restarting it)
2. recreate the cell output

dunno how updating jupyter or the kernel itself could help, but it never hurts 😄

[andreapago]

Hi,
I have just discovered that I have this issue with
jupyterhub 0.6.1
underlying jupyter 4.1.0
R kernel installed following the https://irkernel.github.io/installation/ to what I suppose is the latest version

any idea how to solve it?
thanks

[flying-sheep]

read my first comment in this thread

[jankatins]

Note, the current workaround is adding something like this in the first cell (or at least before any displayed functions or lists/vectors):

library("IRdisplay")
display_latex("\\usepackage[inline]{enumitem}\n\\usepackage{minted}"

There are now two alternative proposals to fix this without the need of a user workaround: IRkernel/repr#74 and #335

[flying-sheep]

wait, you can do that inside of \begin{document}\end{document}?

pretty cool, i didn’t know that!

[bbolker]

bump. I just converted to LaTeX and did a search-and-replace for enumerate* -> enumerate before running pdflatex ... but it would be great to have a general solution

[flying-sheep]

check out #335 and #336

[mpacer]

I'm going to say #335 closes this but will reopen if there it pops up again.