Fix issue with tex-encoding on non-Unicode platforms

[oscargus]

PR Summary

Closes #23019

Thanks @anntzer for the rapid fix.

PR Checklist

Tests and Styling

• Has pytest style unit tests (and pytest passes).
• Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

• [N/A] New features are documented, with examples if plot related.
• [N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
• [N/A] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
• [N/A] Documentation is sphinx and numpydoc compliant (the docs should build without error).

Actually, tests seem to fail?

[oscargus]

Yes, turned out that I didn't check the error message well enough. It raises a CalledProcessError independent of the underlying exception, so I do not really know if it can be tested properly that a RuntimeError (now) is raised rather than a UnicodeDecodeError (earlier).

[anntzer]

I'm a bit confused; it should ultimately raise a RuntimeError, not a CalledProcessError, if you do a fig.canvas.draw(), doesn't it?

[oscargus]

Here is the error message:

In [3]: fig, ax = plt.subplots()
   ...: ax.set_ylabel('\N{SNOWMAN}')
Out[3]: Text(0, 0.5, '☃')Traceback (most recent call last):

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 254, in _run_checked_subprocess
    report = subprocess.check_output(

  File "/local/data1/miniconda3/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

  File "/local/data1/miniconda3/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,

CalledProcessError: Command '['latex', '-interaction=nonstopmode', '--halt-on-error', '../71cab2b5aca12ed5cad4a481b3b00e42.tex']' returned non-zero exit status 1.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/local/data1/miniconda3/lib/python3.8/site-packages/IPython/core/formatters.py", line 341, in __call__
    return printer(obj)

  File "/local/data1/miniconda3/lib/python3.8/site-packages/IPython/core/pylabtools.py", line 151, in print_figure
    fig.canvas.print_figure(bytes_io, **kw)

  File "/local/data1/matplotlib/lib/matplotlib/backend_bases.py", line 2262, in print_figure
    self.figure.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 73, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/figure.py", line 2891, in draw
    mimage._draw_list_compositing_images(

  File "/local/data1/matplotlib/lib/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/axes/_base.py", line 3078, in draw
    mimage._draw_list_compositing_images(

  File "/local/data1/matplotlib/lib/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/axis.py", line 1291, in draw
    self.label.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 687, in draw
    bbox, info, descent = self._get_layout(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 321, in _get_layout
    w, h, d = _get_text_metrics_with_cache(

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 97, in _get_text_metrics_with_cache
    return _get_text_metrics_with_cache_impl(

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 105, in _get_text_metrics_with_cache_impl
    return renderer_ref().get_text_width_height_descent(text, fontprop, ismath)

  File "/local/data1/matplotlib/lib/matplotlib/backends/backend_agg.py", line 235, in get_text_width_height_descent
    w, h, d = texmanager.get_text_width_height_descent(

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 361, in get_text_width_height_descent
    dvifile = cls.make_dvi(tex, fontsize)

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 294, in make_dvi
    cls._run_checked_subprocess(

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 262, in _run_checked_subprocess
    raise RuntimeError(

RuntimeError: latex was not able to process the following string:
b'\\u2603'

Here is the full report generated by latex:
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)
 restricted \write18 enabled.
entering extended mode
(../71cab2b5aca12ed5cad4a481b3b00e42.tex
LaTeX2e <2011/06/27>
Babel <v3.8m> and hyphenation patterns for english, dumylang, nohyphenation, lo
aded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/type1cm/type1cm.sty)
(/usr/share/texlive/texmf-dist/tex/latex/cm-super/type1ec.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/t1cmr.fd))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def
(/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu)))
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty)
(/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty))
(/usr/share/texlive/texmf-dist/tex/latex/underscore/underscore.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu)))
No file 71cab2b5aca12ed5cad4a481b3b00e42.aux.
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd)
*geometry* driver: auto-detecting
*geometry* detected driver: dvips

! Package inputenc Error: Unicode char \u8:☃ not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.29 {\obeylines\sffamily ☃
                             }%
No pages of output.
Transcript written on 71cab2b5aca12ed5cad4a481b3b00e42.log.




<Figure size 432x288 with 1 Axes>

so it seems like the test sees CalledProcessError. I have not figured out a way to detect the underlying exception.

(Somehow one would really like to test that there is not a UnicodeDecodeError rather than an error is raised, in case someone has a latex-setup that actually can do a snowman... So maybe a try-except is better? Although that currently has the same problem.)

[oscargus]

The output above is what I get in Spyder (that automatically does drawing).

Calling fig.canvas.draw() in an ipython console gives the same error message.

[anntzer]

No, the actually thrown exception is the lower one? ("CalledProcessError was the direct cause of the following exception...") Try e.g. wrapping the whole thing in a try... except... to check what exception type you can catch.

[oscargus]

Ahh, you are right. I was confused by the test failure and thought that it raised a different exception, rather than no exception.

[oscargus]

Rebased. Seems like the Ubuntu 3.9-failure somehow is related, but I do not understand if it is or if it is just a random failure that happened to be caused by a test that (maybe) uses the changed code. Edit: restarting the test fixed it.

[oscargus]

Depending on the merge order of this and #23045 I will update the one that is not merged first.

[anntzer]

@oscargus I think you can now import this from matplotlib.testing._markers.

Feel free to self-merge after fixing this.