Can not import matplotlib when launching from non-ascii path

[mdboom]

Can not import matplotlib when launching from non-ascii path. Matplotlib crashes with a UnicodeDecodeError. As far as I can see it is caused by calling os.getcwd and not os.getcwdu when building the path to matplotlibrc.

I could find 4 instances of getcwd in the code base:

__init__.py(748): fname = os.path.join(os.getcwd(), 'matplotlibrc')
backends\backend_gtk.py(820): if not path: path = os.getcwd() + os.sep
backends\backend_gtk3.py(645): if not path: path = os.getcwd() + os.sep
sphinxext\plot_directive.py(451): pwd = os.getcwd()

Example non-ascii path:

In [1]: pwd
Out[1]: u'C:\\python\\external\\\xe5\xe4\xf6'

In [2]: import matplotlib
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-2-82be63b7783c> in <module>()
----> 1 import matplotlib

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in <module>()
   1046
   1047 # this is the instance used by the matplotlib classes
-> 1048 rcParams = rc_params()
   1049
   1050 if rcParams['examples.directory']:

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in rc_params(fail_on_error)
    895     default matplotlib rc file.
    896     """
--> 897     fname = matplotlib_fname()
    898     if not os.path.exists(fname):
    899         # this should never happen, default in mpl-data should always be found

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in matplotlib_fname()
    746       system-defined copy.
    747     """
--> 748     fname = os.path.join(os.getcwd(), 'matplotlibrc')
    749     if os.path.exists(fname):
    750         return fname

C:\Python27-64\lib\ntpath.pyc in join(a, *p)
    106                     path += b
    107                 else:
--> 108                     path += "\\" + b
    109             else:
    110                 # path is not empty and does not end with a backslash,

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 19: ordinal not in range(128)

Ex: ascii path:

In [1]: pwd
Out[1]: u'C:\\python\\external'

In [2]: import matplotlib

In [3]: matplotlib.v
matplotlib.validate_backend matplotlib.verbose

In [3]: matplotlib.__v
matplotlib.__version__        matplotlib.__version__numpy__

In [3]: matplotlib.__version__
Out[3]: u'1.4.0'

[mdboom]

Since this is kind of Windows-specific, maybe you wouldn't mind testing a theory for me. Does changing these calls to os.getcwdu() help?

[jstenar]

Yes that fixes my problem.

[ifmihai]

Since upgrading to 1.4.0, I get the same exception.
I'm on Mac, Mavericks, latest version. anaconda bundle.

I'm not even sure it's about matplotlib, but it crashes when it gets to font_manager

This is the relevant output for the exception:

---> 18 import matplotlib.pyplot as plt
     19
     20 from copy import deepcopy

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/pyplot.py in <module>()
     25
     26 import matplotlib
---> 27 import matplotlib.colorbar
     28 from matplotlib import style
     29 from matplotlib import _pylab_helpers, interactive

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/colorbar.py in <module>()
     32 import matplotlib.artist as martist
     33 import matplotlib.cbook as cbook
---> 34 import matplotlib.collections as collections
     35 import matplotlib.colors as colors
     36 import matplotlib.contour as contour

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/collections.py in <module>()
     25 import matplotlib.artist as artist
     26 from matplotlib.artist import allow_rasterization
---> 27 import matplotlib.backend_bases as backend_bases
     28 import matplotlib.path as mpath
     29 from matplotlib import _path

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/backend_bases.py in <module>()
     54
     55 import matplotlib.tight_bbox as tight_bbox
---> 56 import matplotlib.textpath as textpath
     57 from matplotlib.path import Path
     58 from matplotlib.cbook import mplDeprecation

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/textpath.py in <module>()
     17 from matplotlib.path import Path
     18 from matplotlib import rcParams
---> 19 import matplotlib.font_manager as font_manager
     20 from matplotlib.ft2font import FT2Font, KERNING_DEFAULT, LOAD_NO_HINTING
     21 from matplotlib.ft2font import LOAD_TARGET_LIGHT

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in <module>()
   1410                 verbose.report("Using fontManager instance from %s" % _fmcache)
   1411         except:
-> 1412             _rebuild()
   1413     else:
   1414         _rebuild()

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in _rebuild()
   1395     def _rebuild():
   1396         global fontManager
-> 1397         fontManager = FontManager()
   1398         if _fmcache:
   1399             pickle_dump(fontManager, _fmcache)

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in __init__(self, size, weight)
   1035         #  Load TrueType fonts and create font dictionary.
   1036
-> 1037         self.ttffiles = findSystemFonts(paths) + findSystemFonts()
   1038         self.defaultFamily = {
   1039             'ttf': 'Bitstream Vera Sans',

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in findSystemFonts(fontpaths, fontext)
    320                     fontfiles[f] = 1
    321
--> 322             for f in get_fontconfig_fonts(fontext):
    323                 fontfiles[f] = 1
    324

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in get_fontconfig_fonts(fontext)
    272         pipe = subprocess.Popen(['fc-list', '--format=%{file}\\n'],
    273                                 stdout=subprocess.PIPE,
--> 274                                 stderr=subprocess.PIPE)
    275         output = pipe.communicate()[0]
    276     except (OSError, IOError):

/Users/ifmichael/anaconda/lib/python2.7/subprocess.pyc in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags)
    708                                 p2cread, p2cwrite,
    709                                 c2pread, c2pwrite,
--> 710                                 errread, errwrite)
    711         except Exception:
    712             # Preserve original exception in case os.close raises.

/Users/ifmichael/anaconda/lib/python2.7/subprocess.pyc in _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, to_close, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)
   1325                         raise
   1326                 child_exception = pickle.loads(data)
-> 1327                 raise child_exception
   1328
   1329

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

[tacaswell]

@ifmihai That is in a different section of code (text lookup rather than rcparam file finding), can you open a new issue please?

[mdboom]

I'm going to attach a PR with a fix for the original issue here, and I agree with @tacaswell that @ifmihai's issue is probably something separate.

[cimarronm]

That won't work for python3 since the function doesn't exist as it doesn't need any special unicode return. I don't think six has any compatibility function for this but what I've seen done for this problem in ipython is

try:
    getcwdu = os.getcwdu
except AttributeError:
    getcwdu = os.getcwd

and then use getcwdu function

[mdboom]

@cimarronm: Good catch. Will update.

[ifmihai]

Is it still necessary to create another issue?

For now i downgraded to 1.3.1 and everything got back to normal.

[tacaswell]

@ifmihai Yes, it makes it easier for us to keep track of the bugs in the library.