Animation.save leads to unicode decode error.

[Tillsten]

Running any kind of animation example with ani.save leads to following stacktrace (windows 8.1, mpl 1.4.2, python 2.7 64 bit):

ani = animation.FuncAnimation(fig, animate, np.arange(1, len(y)),
    interval=25, blit=True, init_func=init)

ani.save('double_pendulum.mp4', fps=15)
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-8-34073d277633> in <module>()
     85     interval=25, blit=True, init_func=init)
     86 
---> 87 ani.save('double_pendulum.mp4', fps=15)
     88 #plt.show()

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs)
    750                     #TODO: Need to see if turning off blit is really necessary
    751                     anim._draw_next_frame(d, blit=False)
--> 752                 writer.grab_frame(**savefig_kwargs)
    753 
    754         # Reconnect signal for first draw if necessary

C:\Anaconda\lib\contextlib.pyc in __exit__(self, type, value, traceback)
     22         if type is None:
     23             try:
---> 24                 self.gen.next()
     25             except StopIteration:
     26                 return

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in saving(self, *args)
    175         self.setup(*args)
    176         yield
--> 177         self.finish()
    178 
    179     def _run(self):

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in finish(self)
    356         # are available to be assembled.
    357         self._run()
--> 358         MovieWriter.finish(self)  # Will call clean-up
    359 
    360         # Check error code for creating file here, since we just run

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in finish(self)
    194     def finish(self):
    195         'Finish any processing for writing the movie.'
--> 196         self.cleanup()
    197 
    198     def grab_frame(self, **savefig_kwargs):

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in cleanup(self)
    366 
    367     def cleanup(self):
--> 368         MovieWriter.cleanup(self)
    369 
    370         #Delete temporary files

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in cleanup(self)
    230                        'Command stdout:\n%s' % out, level='debug')
    231         verbose.report('MovieWriter -- '
--> 232                        'Command stderr:\n%s' % err, level='debug')
    233 
    234     @classmethod

UnicodeDecodeError: 'ascii' codec can't decode byte 0x84 in position 5: ordinal not in range(128)

[WeatherGod]

Interesting, this is merely displaying output from the movie saver. What
writer are you using? By default it is ffmpeg, I think. Is that installed
on your system?

On Sun, Nov 23, 2014 at 6:20 AM, Till Stensitzki notifications@github.com
wrote:

Running any kind of animation example with ani.save leads to following
stacktrace (windows 8.1, mpl 1.4.2, python 2.7 64 bit):

ani = animation.FuncAnimation(fig, animate, np.arange(1, len(y)),
interval=25, blit=True, init_func=init)

ani.save('double_pendulum.mp4', fps=15)---------------------------------------------------------------------------UnicodeDecodeError Traceback (most recent call last) in ()
85 interval=25, blit=True, init_func=init)
86 ---> 87 ani.save('double_pendulum.mp4', fps=15)
88 #plt.show()

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs)
750 #TODO: Need to see if turning off blit is really necessary
751 anim._draw_next_frame(d, blit=False)--> 752 writer.grab_frame(**savefig_kwargs)
753
754 # Reconnect signal for first draw if necessary

C:\Anaconda\lib\contextlib.pyc in exit(self, type, value, traceback)
22 if type is None:
23 try:---> 24 self.gen.next()
25 except StopIteration:
26 return

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in saving(self, _args)
175 self.setup(_args)
176 yield--> 177 self.finish()
178
179 def _run(self):

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in finish(self)
356 # are available to be assembled.
357 self._run()--> 358 MovieWriter.finish(self) # Will call clean-up
359
360 # Check error code for creating file here, since we just run

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in finish(self)
194 def finish(self):
195 'Finish any processing for writing the movie.'--> 196 self.cleanup()
197
198 def grab_frame(self, **savefig_kwargs):

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in cleanup(self)
366
367 def cleanup(self):--> 368 MovieWriter.cleanup(self)
369
370 #Delete temporary files

C:\Anaconda\lib\site-packages\matplotlib\animation.pyc in cleanup(self)
230 'Command stdout:\n%s' % out, level='debug')
231 verbose.report('MovieWriter -- '--> 232 'Command stderr:\n%s' % err, level='debug')
233
234 @classmethod
UnicodeDecodeError: 'ascii' codec can't decode byte 0x84 in position 5: ordinal not in range(128)

—
Reply to this email directly or view it on GitHub
#3837.

[Tillsten]

I put the executable in the same directory as the ipynb file, so it should be useable.
But even if its not on the right path, the error msg is not very helpful.

[WeatherGod]

what is the error message if the executable is not in the correct location?
Also, are there any non-ascii characters in the pathname for this
directory? Lastly, have you tried to manually create a movie file using the
ffmpeg executable?

On Sun, Nov 23, 2014 at 9:10 AM, Till Stensitzki notifications@github.com
wrote:

I put the executable in the same directory as the ipynb file, so it should
be useable.
But even if not the error msg is not very helpful.

—
Reply to this email directly or view it on GitHub
#3837 (comment)
.

[Tillsten]

After puting the exe in a folder in sys.path, the everything works, so the error problay was that
in couldnt find ffmpeg.

[WeatherGod]

Strange that it would error out that way, though. There are checks early on
to see what writers are available. Could you run your script without ffmpeg
in your sys.path with the "--verbose-helpful" flag? It should print out a
bunch of debug information that might be useful.

On Sun, Nov 23, 2014 at 9:19 AM, Till Stensitzki notifications@github.com
wrote:

After puting the exe in a folder in sys.path, the everything works, so the
error problay was that
in couldnt find ffmpeg.

—
Reply to this email directly or view it on GitHub
#3837 (comment)
.

[efiring]

On line 232 shown in the traceback, it looks like the problem is that err is a byte string being implicitly converted to unicode using the ascii codec, and it is failing. If nothing else, err needs to be wrapped in an explicit decode, maybe with the Windows codec, and with the errors set to replace instead of the default strict.

[efiring]

Alternatively, all of the Popen initializations could be done with universal_newlines=True, in which case stdout and stderr would be handled as text rather than bytes. That might require other changes; I haven't looked.

[efiring]

I don't think we want to use universal_newlines=True, because it applies to stdout and stderr at the same time, and I think stdout is being used here for a byte stream as well as for text. The simplest thing would be to make the decoding use the same codec as would occur with universal_newlines---except that it is actually version-dependent. (It at least changed between 3.3 and 3.4.) I suspect we should follow the python3.4 model, with locale.getdefaultencoding(False). (The False means it does not call setlocale() first.)
@jbmohler, since this seems to be mainly a Windows problem, would you be able to provide a PR, please?

[jbmohler]

I'm unable to reproduce the bug -- the mp4 writes correctly with both imagemagick & mencoder after I got all the incantations correct (agreed with the OP that the error messages are fairly daunting -- I've never done animations before). However, I'm on master with 32bit (win7) so it's not a direct comparison. I'm going to sleep on it.

I can't really fathom though why we should be getting non-ascii output in stderr from any encoder.

[jbmohler]

Hmm, well I guess the OP agrees that it works once one gets the path incantations correct so I guess this boils down to improving the error messages. The failure I'm getting is not at all like the OP's failure though:

C:\ext\matplotlib\examples\animation>python .\basic_example.py
c:\python27\lib\site-packages\matplotlib\animation.py:728: UserWarning: MovieWriter ffmpeg unavailable
  warnings.warn("MovieWriter %s unavailable" % writer)
Traceback (most recent call last):
  File ".\basic_example.py", line 20, in <module>
    line_ani.save('lines_im.mp4')
  File "c:\python27\lib\site-packages\matplotlib\animation.py", line 753, in save
    writer.grab_frame(**savefig_kwargs)
  File "c:\python27\lib\contextlib.py", line 24, in __exit__
    self.gen.next()
  File "c:\python27\lib\site-packages\matplotlib\animation.py", line 177, in saving
    self.finish()
  File "c:\python27\lib\site-packages\matplotlib\animation.py", line 366, in finish
    + ' Try running with --verbose-debug')
RuntimeError: Error creating movie, return code: 4 Try running with --verbose-debug

[efiring]

@jbmohler, I suspect that if the OS locale is not an English language one, then the OS might return error messages with non-ascii characters in them. On Linux or OSX, I think these would be encoded in UTF-8, but I don't know what Windows does. The error is being returned via stderr as a byte string, and this should be explicitly decoded to unicode prior to being used in the formatted string, using a fail-safe option and a reasonable shot at the encoding. Same for stdout when it is being interpreted as text.

As you say, the point here is not to solve the original problem, but to fix what I believe to be broken error reporting on 4 lines in animation.py, if I remember correctly.

On a Windows machine, what do you get from

import locale
print(locale.getpreferredencoding(False))
print(locale.getpreferredencoding(True))

I think what will be needed is

import locale
_encoding = locale.getpreferredencoding(True)

at the top, and then

out = out.decode(_encoding, 'replace')
err = err.decode(_encoding, 'replace')

prior to using them in error reporting.

[tacaswell]

Using 'replace' apparently has issues on 2.6 #3850 Never mind, I was confused.

[tacaswell]

punting to 1.4.x, any protest?

[petehuang]

Seems like a pretty standalone fix, can we mark it as new-contributor-friendly?

[efiring]

@Tillsten would you check to see whether the problem still exists, and if it does, then see whether my suggestion works, and if it works, then make a PR for it? Thank you.

[tacaswell]

I also suspect that this will 'just work' in python3

[Tillsten]

Sorry, no Py2 avaible anymore :)

[anntzer]

I think this has almost certainly been fixed with the switch to logging and a bunch of fixes to animation logging on similar issues as well. Closing, but feel free to reopen with a repro.