from matplotlib import animation UnicodeDecodeError

[tomdean1939]

Bug report

Bug summary

In interactive python3 'from matplotlib import animation' causes an error.
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 87: ordinal not in range(128)

Code for reproduction

from matplotlib import animation

Actual outcome

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/animation.py", line 737, in <module>
    class ImageMagickWriter(ImageMagickBase, MovieWriter):
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/animation.py", line 120, in wrapper
    if writerClass.isAvailable():
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/animation.py", line 730, in isAvailable
    return super().isAvailable()
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/animation.py", line 427, in isAvailable
    return shutil.which(cls.bin_path()) is not None
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/animation.py", line 724, in bin_path
    binpath = mpl._get_executable_info('magick').executable
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/__init__.py", line 384, in _get_executable_info
    return impl([path, "--version"], r"^Version: ImageMagick (\S*)")
  File "/usr/local/lib/python3.6/dist-packages/matplotlib/__init__.py", line 324, in impl
    args, stderr=subprocess.STDOUT, universal_newlines=True)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 425, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.6/subprocess.py", line 850, in communicate
    stdout = self.stdout.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 87: ordinal not in range(128)

Expected outcome
matplotlib.animation would be imported.

I am not sure it ever worked.

Matplotlib version

• Operating system: Ubuntu 18.04
• Matplotlib version: 3.1.1
• Matplotlib backend (print(matplotlib.get_backend())): GTK3Agg
• Python version: 3.6.8
• Jupyter version (if applicable):
• Other libraries:

Python3 came with the distribution and was updated with apt
matplotlib was installed with python3 -m pip install matplotlib. Updated with pip.

[ImportanceOfBeingErnest]

Might be related #3837

Possibly you will get more information with

import logging
logging.basicConfig(level=0)

[jklymak]

Not sure this error makes any sense regardless, but do you have imagemagick installed?

[tomdean1939]

This error stops animation from being import'd.
And, it is related to imagemagick. Looking at python3.6/encodings/ascii.py, the 'input' that causes the error contains: b'Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114...

My log does not show imagemagick being installed. Must have either come with the system or some other package required it.
How does imagemagick get involved?

[jklymak]

In order to run an animation we call outside packages. You need imagemagick if you use the imagemagick movie writer

OTOH this error probably shouldn’t happen. Ping @antzer who I think did some work on this recently

[ImportanceOfBeingErnest]

Here is a pretty similar issue, https://stackoverflow.com/questions/57115893, though with a different error.

Clearly, the problem is that

matplotlib/lib/matplotlib/animation.py

Line 728 in 8d3e817

except mpl.ExecutableNotFoundError:

only checks for some "executable not found", but not for other errors that could happen along the way.

[ImportanceOfBeingErnest]

Probably #13303 is now causing this to come up. Ping @anntzer (the one with double-N ;-))

[tomdean1939]

I see the problem, but, not how to fix it. In determining the ImageMagick version, I think matplotlib executes 'convert --version' convert --version Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org Copyright: © 1999-2017 ImageMagick Studio LLC License: http://www.imagemagick.org/script/license.php Features: Cipher DPC Modules OpenMP Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jpeg lcms lqr ltdl lzma openexr pangocairo png tiff wmf x xml zlib Notice the Copyright sign, after the word Copyright. This is what codecs pukes on. I have sys.getdefaultencoding() returns utf-8.

[ImportanceOfBeingErnest]

This makes this issue two-fold:

1. Even if trying to run the executable were to return total non-sense, the respective writer should just not be registered instead of erroring out. More logging would also help here.
2. Apparently some versions of imagemagick report different strings back. Of course this should not hinder the use of imagemagick (or in fact any other external tool matplotlib may want to use).

[tomdean1939]

This patch makes it work. Don't know what other things it may break... *** /usr/lib/python3.6/encodings/ascii.py.orig Sat Jul 27 14:57:32 2019 --- /usr/lib/python3.6/encodings/ascii.py Sat Jul 27 15:04:34 2019 *************** *** 23,28 **** --- 23,30 ---- class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): + nonascii = bytearray(range(0x80, 0x100)) + input = input.translate(None, nonascii) return codecs.ascii_decode(input, self.errors)[0] class StreamWriter(Codec,codecs.StreamWriter):

[anntzer]

I guess something like

diff --git i/lib/matplotlib/__init__.py w/lib/matplotlib/__init__.py
index 8103bc3d6..99b24483e 100644
--- i/lib/matplotlib/__init__.py
+++ w/lib/matplotlib/__init__.py
@@ -330,7 +330,8 @@ def _get_executable_info(name):
         # at least min_ver (if set); else, raise ExecutableNotFoundError.
         try:
             output = subprocess.check_output(
-                args, stderr=subprocess.STDOUT, universal_newlines=True)
+                args, stderr=subprocess.STDOUT,
+                universal_newlines=True, errors="replace")
         except subprocess.CalledProcessError as _cpe:
             if ignore_exit_code:
                 output = _cpe.output

would work?

[tomdean1939]

That seems to have fixed it on my machine. Possibly a better fix, higher in the process change and a less intrusive change.