LaTeX support is broken

[flying-sheep]

In our IRkernel, we use repr to create mimebundles. this includes LaTeX and HTML for many things that aren’t plots.

This results in QtConsole displaying PNGs with LaTeX source code in them instead of something useful.

Please prefer the HTML version if available. Also LaTeX doesn’t necessarily mean “equation”.

[minrk]

Of the HTML output we've seen, the qtconsole supports so little of it, that we've disabled HTML output in the qtconsole. Unfortunately, we don't have a good way to distinguish "bare minimum subset of HTML that Qt supports" vs "actual HTML with javascript and the whole shebang for browsers". If you know of a good way to specify that, it would be helpful. Perhaps there is a mimetype for text/x-html-but-only-sort-of.

[flying-sheep]

the way LaTeX is used is equally wrong: effectively it also expects a subset.

our kernel provides rich representations of data structures in plain text, Markdown, HTML, and LaTeX, and rich representations of plots in PDF, PNG, JPG, and SVG.

i think the qt console should handle that.

[minrk]

The QtConsole cannot handle HTML in general until it is rewritten to use a web view instead of the QtRichTextWidget that it's currently based on.

[flying-sheep]

as i said: it also cannot handle LaTeX in general

[flying-sheep]

so in order to be consistent:

if you removed HTML handling, you should also remove LaTeX handling.

and if you want to keep LaTeX handling, you should also add HTML handling.

[minrk]

Then I suppose we should remove latex.

[flying-sheep]

it was added in cdf5ead as “Add Math display support for qtconsole”, so yeah: definitely expects a subset, so if we ony want to includes mimetypes that are fully handled, we should remove latex.

[minrk]

I agree that treating latex and html equally is a sensible plan. We should probably at least allow users to turn them on/off, whichever we end up choosing as the default, since there is a small set of HTML outputs that might work.

[ccordoba12]

I added support for Latex in qtconsole. Latex is shown as png images, so this doesn't have the same problem as rendering Html in a QTextEdit widget.

If no Latex distribution is installed, Qtconsole uses Matplotlib Latex engine as a fallback. I'm sure Matplotlib doesn't support all Latex constructs, but it's still pretty useful. Please don't remove support for it.

[minrk]

@ccordoba12 thanks, so our latex support is actually better than our HTML support.

[flying-sheep]

ah, so it will use pdflatex if it can find it? the machinery to find it is not very powerful then: i have a pretty normal linux setup.

but as said: it’s not useful if your kernel emits stuff it can’t handle:

better is apparently not good enough. any ideas how to fix this?

[flying-sheep]

anything that uses the latex machinery to render our latex output will be subar for qtconsole:

culry font, uncopyable text, slow due to calling subprocesses

[flying-sheep]

i still think we need to fix it here, because of the hackiness of fixing it in IRkernel.

that would be basically “let’s execute code conditionally depending on the quality in which frontends render specific output formats” 😷

[ccordoba12]

SymPy also uses Latex printing in qtconsole, and we have an option in Spyder to activate it.

---

anything that uses the latex machinery to render our latex output will be subar for qtconsole

I understand that, but it's something that qtconsole can really handle, as opposed to html, which qtconsole couldn't do it in all cases.

[flying-sheep]

so what’s your idea to fix this problem?

because while i can see the practical value, i also see that it’s not full support. and without full support, it will do buggy things.

[ccordoba12]

so what’s your idea to fix this problem?

Just fix it. I don't understand why it's not working for IRKernel. Besides, as I said, it uses pdflatex in the background (if it's found), so there's full Latex support already.

[flying-sheep]

it won’t. the limited matplotlib implementation is used by default, not latex.

we’ll need to specify backend='dvipng' in the latex2png call, and we’ll need to implement the fallback ourself (i.e. find out what exceptions are rised if the binary isn’t found or if it fails to run, catch those and try again with the other backend)

could you do that? i have enough PRs pending as of now and you are interested in keeping that functionality instead of removing it, so i think you should spearhead a correctly working version 😉

to put it in other words: this is a regression since the output after your commit is less useful (for IRkernel, which behaves perfectly in accordance to spec and therefore needs no changes). in most projects i know, regressions are fixed or the corresponding changes reverted.

[ccordoba12]

@minrk, what's your take on @flying-sheep position? I didn't know there was a regression regarding this feature.

[minrk]

If you think switching the latex default to 'real' latex when available works well enough, I'm AOK with that. I suspect the situation is that we should have a config value for each, so people can turn it on or off at will. In my experience, the latex behavior is good enough that it should be on by default and the html is bad enough that it should be off by default, but my experience is not necessarily representative. You have more experience with QtConsole and QtConsole users, so I'm happy to defer to you on this one.

[flying-sheep]

In my experience, the latex behavior is good enough that it should be on by default

now that you’ve seen my screenshot, your experience should have changed to “the LaTeX behavior is not good enough” but feel free to try yourself 😛

I didn't know there was a regression regarding this feature.

now you do. the Qt Console had working plain text display when used with the kernel before that commit. this means that commit introduced a regression.

i think if you fix it by using the dvipng backend like outlined above, that’ll be good enough. sadly we won’t be able to copy text, but at least we don’t get LaTeX code in the Qt console.

PS: don’t think i’m doing this to be obtuse and difficult! i just want this to work for everyone. 😺

[jorisvandenbossche]

We have a similar issue in pandas. On master, recently a _repr_latex_ method was added to DataFrame (for use in nbconvert to pdf). But now, there is also a report of a wrong output in the qtconsole, (as qtconsole tries to use this repr, preferred over the text repr, but qtconsole latex renderer does not support latex tables).
See pandas-dev/pandas#12182

[asmeurer]

@jorisvandenbossche does my recently merged #65 fix your issue?

[jorisvandenbossche]

I didn't test it myself, but I asked the reporter of the issue to test with master and he said it also happens there.

To clarify for myself: pandas DataFrames now have a _repr_latex_ method (in pandas master). And as far as I understand it, the qtconsole will always use this repr if it is defined and not the plain text repr? There is no way to have this method defined (for usage in nbconvert), while letting the qtconsole not make use of it?
Because even if the latex handling would be better and the table would be rendered nicely in latex, I don't think we would want this (converting a dataframe to a latex table will always be a somewhat limiting conversion to represent the actual dataframe)

[flying-sheep]

which is of course what i said the whole time, but sadly was alone with.

it’s pretty stupid to have PNGs of text in a text-based interface.

[asmeurer]

Ah, my PR only handles the case where latex is not installed but matplotlib is (and the latex isn't handled by matplotlib's renderer).

I think what we have here is now at least three projects (SymPy, IRKernel, and Pandas) that would benefit from the ability to set their own Jupyter repr priorities (right now they are hard-coded for each front-end—the qtconsole prefers latex over text and there's nothing you can do about it). So maybe that's impetus enough for the Jupyter developers to actually implement it.

[asmeurer]

I also should point out again that @flying-sheep's PR #57 won't actually break SymPy, because it does its own LaTeX -> png conversion independently.

[ccordoba12]

@asmeurer, that's not the solution because qtconsole is able to handle Latex, as you have proved it with Sympy (but bypassing somehow the repr mechanism).

The real solution is to make Latex support to work correctly, and for Sympy to use it. I think there's no escape to that because Sympy objects come with _repr_latex_ (to be displayed in the notebook I guess), so the current methods to handle latex in qtconsole are applied to them too. That's why you had to create PR #63 :-)

I'm working on a fix right now.

---

@jorisvandenbossche, the problem is that df.to_latex() generates the latex representation of a DataFrame, but doesn't enclose it in $...$ or $$...$$ and qtconsole expects that to work correctly. I'm almost sure that's the same problem observed in IRKernel too.

@minrk, how the notebook handles latex? Does _repr_latex_ need to be enclosed in dollar signs or not? I guess that not.

---

it’s pretty stupid to have PNGs of text in a text-based interface.

Well, there's no other way to handle latex in qtconsole. MathJax won't work here because the amount of html that QTextEdit widgets can handle is pretty limited. That's why html support was removed some versions ago.

[asmeurer]

Is there some way to render the latex as a pdf or svg, so that the text is actually selectable?

[jorisvandenbossche]

@jorisvandenbossche, the problem is that df.to_latex() generates the latex representation of a DataFrame, but doesn't enclose it in $...$ or $$...$$ and qtconsole expects that to work correctly. I'm almost sure that's the same problem observed in IRKernel too.

Possibly that is the issue here (but, a table environment is not an equation, so from a latex point of view should not be wrapped in $. So the latex repr support is maybe more a 'latex equation/math env' support). But anyhow, as I said above, we do not want to use this latex repr for dataframes in the qtconsole. So for now, the only option to have that behaviour is by not defining a repr_latex I think?

[flying-sheep]

Sympy objects come with repr_latex (to be displayed in the notebook I guess)

probably to display in the PDF created by nbconvert. just as IRkernel’s latex representations.

qtconsole is able to handle Latex

not really, unless it can convert it into selectable and copyable rich text (hint: don’t even try)

Well, there's no other way to handle latex in qtconsole.

there is: we could make qtconsole check if the latex representation is wrapped in $ or $$or \equation or whatever there is. if it is, we try to convert it, else we use another representation that’ll be converted to actual rich text

[ccordoba12]

if it is, we try to convert it

convert it to what?

else we use another representation that’ll be converted to actual rich text

There is no rich text in qtconsole because html support for it was removed and it won't come back (as I said above).

[asmeurer]

probably to display in the PDF created by nbconvert. just as IRkernel’s latex representations.

No, the SymPy LaTeX printing really is what we want. See http://docs.sympy.org/latest/tutorial/printing.html. Note that SymPy objects don't have _repr_latex_ by default. It is only added after running init_printing() (and it can be prevented from being added by running init_printing(use_latex=False)). This was done because it's the only way for the user to be able to choose a different kind of printing if he wants it, since Jupyter doesn't allow re-ordering them (which, I'll reiterate, is what I think the real problem is here).

[flying-sheep]

convert it to what?

via LaTeX/matplotlib to PNGs that we display. we won’t be able to select text, but it will only be for stuff that’s intended as equation.

There is no rich text in qtconsole because html support for it was removed and it won't come back (as I said above).

OK then normal text representation then. doesn’t matter

[ccordoba12]

probably to display in the PDF created by nbconvert. just as IRkernel’s latex representations.

This shows a common trend (along with Pandas): adding repr_latex just for nbconvert.

---

the only option to have that behaviour is by not defining a repr_latex I think?

@jorisvandenbossche, I don't think that's the option. I think the solution is what @asmeurer is saying:

Jupyter doesn't allow re-ordering repr's (which, I'll reiterate, is what I think the real problem is here).

I can't think of a better alternative.

---

@minrk, your thoughts?

[jorisvandenbossche]

There is no rich text in qtconsole because html support for it was removed and it won't come back (as I said above).

But there is plain text? It is in any case richer as the current latex repr.

I think it is clear that the current latex support of qtconsole is limited to mathematical expression, is that correct?
But, a lot of objects can have reprs that are more than that, eg tables. So I think it would be a good idea to restrict the latex support in the qtconsole to only mathematical expressions, as @flying-sheep proposes (but not sure how to easily make this distinction)

[flying-sheep]

No, the SymPy LaTeX printing really is what we want.

with my proposal that would be fine. just serve stuff between $$ and you’ll be good.

[jorisvandenbossche]

@jorisvandenbossche, I don't think that's the option. I think the solution is what @asmeurer is saying:

Jupyter doesn't allow re-ordering them repr's.

I can't think of a better alternative.

Yes, but that is for a possible future version of qtconsole then? That will not help us solve this issue for the upcoming pandas release I think.

[ccordoba12]

I think it is clear that the current latex support of qtconsole is limited to mathematical expression, is that correct?

Nop, if you have Latex installed, it can render arbitrary complex expressions (as Sympy proves it). What we can do is not to use matplotlib to try to render latex because its support is very limited.

[ccordoba12]

@flying-sheep, what rich text representation are you talking about? I really don't understand it.

[flying-sheep]

but not sure how to easily make this distinction

heuristically. we simply detect strings that are wrapped by one of the environments that enable mathmode, e.g. $x$, $$x$$, \(x\), \begin{equation}x\end{equation}, ...

if you have Latex installed, it can render arbitrary complex expressions (as Sympy proves it)

which nobody wants, because text will be preferable in those cases.

what rich text representation are you talking about? I really don't understand it.

i meant text/html mimebundles. but if qtconsole only supports text/plain that’s still better than PNG-rendered LaTeX for non-equations

[jorisvandenbossche]

I think it is clear that the current latex support of qtconsole is limited to mathematical expression, is that correct?

Nop, if you have Latex installed, it can render arbitrary complex expressions (as Sympy proves it). What we can do is not to use matplotlib to try to render latex because its support is very limited.

@ccordoba12 Are you sure? I have latex installed, but have the exact same problem (a png with latex \begin{tabular} code). Also when I wrap the return value in $ or $$, I also get an error message

[ccordoba12]

i meant text/html mimebundles. but if qtconsole only supports text/plain that’s still better than PNG-rendered LaTeX

Now I understand you!! So if things don't come enclosed in dollar signs we skip Latex and move to plain text. That should work for Pandas too! :-)

@asmeurer, are you ok with that?

[jorisvandenbossche]

Now I understand you!! So if things don't come enclosed in dollar signs we skip Latex and move to plain text. That should work for Pandas too! :-)

Indeed, that's how I understood @flying-sheep's proposal, and I would be +1 to that

[ccordoba12]

Sorry, I got confused by his insistence on rich text :-p But yes, it's a pretty good proposal. Let's wait to see if it works for Sympy too.

[ccordoba12]

It seems all Sympy objects come enclosed in dollar signs, but PR #63 along with a couple of fixes I plan to add should take care of its case.

[flying-sheep]

OK, great. stuff that triggers mathmode:

inline:

• \begin{math}...\end{math}
• \(...\)
• $...$

display:

• \begin{displaymath}...\end{displaymath}
• \[...\]
• $$...$$
• \begin{equation}...\end{equation}
• \begin{equation*}...\end{equation*}
• those

[asmeurer]

I don't know if it will break SymPy. You'll have to try it and test it out. Just try running

from sympy import *
init_printing()
sqrt(2)
Matrix([[1, 2], [3, 4]])

in the qtconsole and in the notebook (if the changes would be affected there). Also worth running

from sympy import *
init_printing(use_latex=False)
sqrt(2)
Matrix([[1, 2], [3, 4]])

And also make note whether matplotlib and/or latex is installed (there are a lot of combinations to test here).