UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

[javadr]

Bug report

When I want to use pyplot from matplotlib, it causes error.
There is no problem with import matplotlib but I cannot use pyplot

Code for reproduction

from matplotlib import pyplot

Actual outcome

---------------------------------------------
FileNotFoundErrorTraceback (most recent call last)
/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in <module>()
   1428         try:
-> 1429             fontManager = pickle_load(_fmcache)
   1430             if (not hasattr(fontManager, '_version') or

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in pickle_load(filename)
    964     """
--> 965     with open(filename, 'rb') as fh:
    966         data = pickle.load(fh)

FileNotFoundError: [Errno 2] No such file or directory: '/home/javad/.cache/matplotlib/fontList.py3k.cache'

During handling of the above exception, another exception occurred:

UnicodeDecodeErrorTraceback (most recent call last)
<ipython-input-3-a371ea114746> in <module>()
----> 1 from matplotlib import pyplot

/usr/lib64/python3.6/site-packages/matplotlib/pyplot.py in <module>()
     27 from cycler import cycler
     28 import matplotlib
---> 29 import matplotlib.colorbar
     30 from matplotlib import style
     31 from matplotlib import _pylab_helpers, interactive

/usr/lib64/python3.6/site-packages/matplotlib/colorbar.py in <module>()
     34 import matplotlib.collections as collections
     35 import matplotlib.colors as colors
---> 36 import matplotlib.contour as contour
     37 import matplotlib.cm as cm
     38 import matplotlib.gridspec as gridspec

/usr/lib64/python3.6/site-packages/matplotlib/contour.py in <module>()
     20 import matplotlib.colors as colors
     21 import matplotlib.collections as mcoll
---> 22 import matplotlib.font_manager as font_manager
     23 import matplotlib.text as text
     24 import matplotlib.cbook as cbook

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in <module>()
   1437             raise
   1438         except:
-> 1439             _rebuild()
   1440     else:
   1441         _rebuild()

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in _rebuild()
   1416         global fontManager
   1417 
-> 1418         fontManager = FontManager()
   1419 
   1420         if _fmcache:

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in __init__(self, size, weight)
   1070         self.afmfiles = findSystemFonts(paths, fontext='afm') + \
   1071             findSystemFonts(fontext='afm')
-> 1072         self.afmlist = createFontList(self.afmfiles, fontext='afm')
   1073         if len(self.afmfiles):
   1074             self.defaultFont['afm'] = self.afmfiles[0]

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in createFontList(fontfiles, fontext)
    581             try:
    582                 try:
--> 583                     font = afm.AFM(fh)
    584                 finally:
    585                     fh.close()

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in __init__(self, fh)
    347         """
    348         (dhead, dcmetrics_ascii, dcmetrics_name, dkernpairs, dcomposite) = \
--> 349             parse_afm(fh)
    350         self._header = dhead
    351         self._kern = dkernpairs

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in parse_afm(fh)
    335     _sanity_check(fh)
    336     dhead = _parse_header(fh)
--> 337     dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
    338     doptional = _parse_optional(fh)
    339     return dhead, dcmetrics_ascii, dcmetrics_name, doptional[0], doptional[1]

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in _parse_char_metrics(fh)
    196         if not line:
    197             break
--> 198         line = line.rstrip().decode('ascii')  # Convert from byte-literal
    199         if line.startswith('EndCharMetrics'):
    200             return ascii_d, name_d

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

Expected outcome

using %matplotlib magic in jupyter notebook also produces the same error.

Matplotlib version

• Operating System: Fedora 26, 64bit
• Matplotlib Version: 2.0.0-2
• Python Version: 2.7 / 3.6
• Jupyter Version (if applicable): 4.1.0
• Other Libraries:

[tacaswell]

This is a duplicate of #3517 and which is fixed by #8021 which is in v2.0.1

[tacaswell]

@javadr Sorry you are having this issue and thank you for reporting it. This should be reported to Fedora to update their packaged version.

[javadr]

Appreciate your concern and your fast reply.

I updated the matplotlib to 2.1.0rc1 with pip, but the problem not solved yet. :(

[WeatherGod]

Most likely, you have to clear out the old installation and reinstall the new version.

…

On Mon, Sep 18, 2017 at 3:43 AM, javadr ***@***.***> wrote: Appreciate your concern and your fast reply. I updated the matplotlib to 2.1.0rc1 with pip, but the problem not solved yet. :( — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9196 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-Mm82ILp2rXYOk5Vx-ClPd69WHNMks5sjh81gaJpZM4PaITb> .

[tacaswell]

@javadr can you drop into the debugger and determine which file it is failing on?

I was wrong, this is a different bug and the this decode is still on line 192 in afm.py

[javadr]

thanks again, I hope that the following output help you.

Python 2.7.13 (default, Sep  5 2017, 08:53:59) 
[GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> import matplotlib.pyplot

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/usr/lib64/python2.7/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/usr/lib64/python2.7/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

P.S.
There is no difference between python 2.7 and 3.6! both makes same results!

Python 3.6.2 (default, Aug 11 2017, 11:59:59) 
[GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] on linux
Type "copyright", "credits" or "license()" for more information.
>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1429, in <module>
    fontManager = pickle_load(_fmcache)
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 965, in pickle_load
    with open(filename, 'rb') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/home/javad/.cache/matplotlib/fontList.py3k.cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot
  File "/usr/lib64/python3.6/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/usr/lib64/python3.6/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/usr/lib64/python3.6/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

[javadr]

This is so weird, I've also installed Anaconda2 and I've got the same error!

Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

I don't know is it something wrong with fedora 26 or not?!

[javadr]

I found the problem, It was because of my installed fonts! I removed some of them and now I can use matplotlib without any problem. As I noticed some of my fonts were duplicates! (Two distinct directories contained the same set of fonts, so I deleted one of them and it solved the problem.)

Sorry to cause you inconvenience.

[tacaswell]

@javadr which font (that is what I meant by 'which file' above, sorry for being unclear!)?

I am interested in looking at exactly what it has inside to understand why this is happening. We should be able to handle, or at least gracefully fail, on (possibly) malformed font!

[javadr]

Unfortunately, I removed many fonts, and now I don't know which one caused the problem , but if I find that problematic font, I'll send it.

[tacaswell]

Thanks @javadr .

Glad things are working for you again!

[MDoYA]

Hello All,
I have apparently the same problem with Pycharm IDE installed on Windows 10 (Anaconda, python 2.7). Matplotlib and utf-8 characters in my path (French) drive the python interpreter in issuing the exact same error as Javadr. Pycharm project and interpreter into it are set on utf-8.
Anyone could tell me if I should try to remove font one by one (how I do that, I did not add any I believe) or another solution has been found?

Thank you

PS : Anaconda is installing also Spyder IDE and the same code works perfectly on it.

operating system :
Matplotlib Version: 2.2.2, also tried 2.1.0, same error
Python Version: 2.7.14
IDE : Pycharm 2018.1.3

[anntzer]

Does it work if you replace the line

    if not line.startswith('  File "<frozen importlib._bootstrap'))

by

    if not line.startswith(b'  File "<frozen importlib._bootstrap'))

?
Or even better perhaps, remove the unicode_literals import at the very top of the file (basically, #10044).

Reopening although the issue is kind of different, but that's fine.

[MDoYA]

Hello anntzer,
I have tried your solution and then I just removed the unicode_literals import like this :

from __future__ import (absolute_import, division, print_function)#,
#                        unicode_literals)

It makes no difference. Just to be accurate, the following pic : is the Error I get if running the script with a ipython console launched from the path window :

Here is the mistake I get in the Python console of Pycharm :

I will try to go further in the comments of #10044

[anntzer]

You edited C:\Anaconda2\lib\site-packages\matplotlib\backends\__init__.py when removing the unicode_literals import, right?

What about the other proposed fix of using b' File ...'?

[MDoYA]

Yes I have firstly edited the script with your proposed fixture using b' File ...'
Then I have edited removing unicode_literals import.
I have restarted the Pycharm IDE inbetween to be sure to initialize properly the Matplotlib librairies.
Both ended with the same UnicodeError.

[AndreySV]

I've got the same problem. On Debian Buster with matplotlib 2.2.2.
The problem is that traceback.format_stack() returns list of 'str' objects. If some of the modules in the stack was located in directory with non-ascii character in path, then such error happens.

To address this problem I recommend to use traceback2, that always returns list of 'unicode'.
It's available in Debian and Ubuntu archives as well (python-traceback2, python3-traceback2) packages and of course in pypi.

Change is simple: replace 'import traceback' with 'import traceback2 as traceback' in matplotlib/backends/init.py

[tacaswell]

The code path in question no longer exists on master branch / 3.0 branch. This is a python2 only issue and mpl3.0+ will not support python2.

@AndreySV I am skeptical of picking up a new dependency on a micro release (ex 2.2.4). Does

_backend_loading_tb = b"".join(
    line for line in traceback.format_stack()
    # Filter out line noise from importlib line.
    if not line.startswith(b'  File "<frozen importlib._bootstrap'))

fix the problem? In either case, can you open a new issue for this issue which is an entirely different issue than the original.

[AndreySV]

Yes, convert to bytes both strings helps. I've opened for that separate issue #11955

[tacaswell]

I'm going to re-close this it is really two issues, the first of which is already fixed and the second of which now has it's own issue.

[shanglidan]

Before import matplotlib.pyplot, you can use the following code.
import sys
reload(sys)
sys.setdefaultencoding('utf8')