Improvement suggestions for matplotlib.Animation.save('video.mp4') #3865

Closed
Zulko opened this Issue Nov 29, 2014 · 5 comments

Comments

Projects
None yet
5 participants
@Zulko

Zulko commented Nov 29, 2014

TLDR : Maybe it would be possible to speed up 2x matplotlib's video writing by sending directly a raw RGB input to the writer (ffmpeg, avconv). Also, Matplotlib.Animation's default encoding is bad.

Hi there,

I compared writing a video using Matplotlib's animation module, with writing the same Matplotlib video using my library MoviePy instead of Animation.save().

See here for details and results.

It turns out that MoviePy produces an animation of better quality (Matplotlib's has
JPEG-compression-style artefacts) and much lighter (95k for MoviePy, 456k for Matplotlib).
MoviePy is also much faster (generating the animation takes 6 seconds with MoviePy, against 12s
for Matplotlib).

For the quality/size, it may be a matter of codecs or whatever (I really can't find what), I'm sure it can be tuned.

For the speed, however, it may be due to the fact that Matplotlib converts the frame into some compressed image format (I believe it is PNG by default) and sends them to the writer (ffmpeg in my case), which then must decode it. MoviePy sends the raw RGB snapshot of the matplotlib canvas directly to ffmpeg (no encoding/decoding needed).

What do you think of that ?

@efiring

This comment has been minimized.

Show comment
Hide comment
@efiring

efiring Nov 29, 2014

Member

A pull request to improve the animation module would be most welcome! It would be good to have a range of test cases so that we can be sure we are not optimizing for one extreme case (e.g. a simple animated line plot) at the expense of another (e.g., contouring or imaging a function of x, y, and t).

Member

efiring commented Nov 29, 2014

A pull request to improve the animation module would be most welcome! It would be good to have a range of test cases so that we can be sure we are not optimizing for one extreme case (e.g. a simple animated line plot) at the expense of another (e.g., contouring or imaging a function of x, y, and t).

@Zulko

This comment has been minimized.

Show comment
Hide comment
@Zulko

Zulko Nov 29, 2014

Sure. I'm still not sure I have identified the problem, I may give it another look when I have time. The difficulty, I think is to ensure that the "improvement" will not break any of the writers (ffmpeg, avconv, mencoder).

Zulko commented Nov 29, 2014

Sure. I'm still not sure I have identified the problem, I may give it another look when I have time. The difficulty, I think is to ensure that the "improvement" will not break any of the writers (ffmpeg, avconv, mencoder).

@WeatherGod

This comment has been minimized.

Show comment
Hide comment
@WeatherGod

WeatherGod Nov 29, 2014

Member

When @dopplershift and I (well, mostly @dopplershift) worked on the
animation module, we these two concerns were not on our list, so it is good
to see someone else looking into them. We are not movie experts by any
stretch of the imagination. I know there are lots more that can be done to
improve the default settings, and I can certainly help in guiding you to
the right places in the code.

Yes, you are right that PNG's are sent to the encoder. This was a logical
step from the original per-frame file based writer to the direct pipe
writer. PNGs will likely still be needed for the file-based writers, but it
should be feasible to capture the raw rgba buffers instead for the piped
writers. Speedups would always be nice to have.

An important thing to keep in mind. I know that many video enthusiasts like
to squeeze every single bit out of their encodings and get to the smallest
file size possible. For Matplotlib, we optimize for maximum sharing. I
meticulously tested (3 years ago) several different combinations of
containers and codecs to see which ones work on stock Windows, Linux and
Mac installs, particularly for use in PowerPoint and OpenOffice/LibreOffice
Impress. This resulted in the current defaults. Therefore, I am open to
improved file sizes, but we will need to ensure that such choices do not
prevent use of the video on stock installations of these platforms.

I would be more than happy to review your PRs. Cheers!

On Sat, Nov 29, 2014 at 2:18 PM, Zulko notifications@github.com wrote:

Sure. I'm still not sure I have identified the problem, I may give it
another look when I have time. The difficulty, I think is to ensure that
the "improvement" will not break any of the writers (ffmpeg, avconv,
mencoder).


Reply to this email directly or view it on GitHub
#3865 (comment)
.

Member

WeatherGod commented Nov 29, 2014

When @dopplershift and I (well, mostly @dopplershift) worked on the
animation module, we these two concerns were not on our list, so it is good
to see someone else looking into them. We are not movie experts by any
stretch of the imagination. I know there are lots more that can be done to
improve the default settings, and I can certainly help in guiding you to
the right places in the code.

Yes, you are right that PNG's are sent to the encoder. This was a logical
step from the original per-frame file based writer to the direct pipe
writer. PNGs will likely still be needed for the file-based writers, but it
should be feasible to capture the raw rgba buffers instead for the piped
writers. Speedups would always be nice to have.

An important thing to keep in mind. I know that many video enthusiasts like
to squeeze every single bit out of their encodings and get to the smallest
file size possible. For Matplotlib, we optimize for maximum sharing. I
meticulously tested (3 years ago) several different combinations of
containers and codecs to see which ones work on stock Windows, Linux and
Mac installs, particularly for use in PowerPoint and OpenOffice/LibreOffice
Impress. This resulted in the current defaults. Therefore, I am open to
improved file sizes, but we will need to ensure that such choices do not
prevent use of the video on stock installations of these platforms.

I would be more than happy to review your PRs. Cheers!

On Sat, Nov 29, 2014 at 2:18 PM, Zulko notifications@github.com wrote:

Sure. I'm still not sure I have identified the problem, I may give it
another look when I have time. The difficulty, I think is to ensure that
the "improvement" will not break any of the writers (ffmpeg, avconv,
mencoder).


Reply to this email directly or view it on GitHub
#3865 (comment)
.

@tacaswell tacaswell added this to the unassigned milestone Nov 29, 2014

@tacaswell tacaswell added the Animation label Dec 4, 2014

@videobrain

This comment has been minimized.

Show comment
Hide comment
@videobrain

videobrain Aug 13, 2016

Just to add an alternative view about possible use cases: I would like to animate charts, but need high quality video output to combine that with other video material. Compression will happen with the final product at the end of the pipeline, but the output of Matplotlib should be very high quality so the material can be still used in the following video transformation steps.

Just to add an alternative view about possible use cases: I would like to animate charts, but need high quality video output to combine that with other video material. Compression will happen with the final product at the end of the pipeline, but the output of Matplotlib should be very high quality so the material can be still used in the following video transformation steps.

@tacaswell

This comment has been minimized.

Show comment
Hide comment
@tacaswell

tacaswell Aug 13, 2016

Member

Closing this as streaming writer exist in the code base for ffmpeg and libav (ex http://matplotlib.org/api/animation_api.html?highlight=animation#matplotlib.animation.FFMpegWriter). mpl passes the full-resolution RBGA buffer via a pipe to the encoder. Exactly how that is compressed is up to the underlying software (and you can pass through what ever cl args you want).

Member

tacaswell commented Aug 13, 2016

Closing this as streaming writer exist in the code base for ffmpeg and libav (ex http://matplotlib.org/api/animation_api.html?highlight=animation#matplotlib.animation.FFMpegWriter). mpl passes the full-resolution RBGA buffer via a pipe to the encoder. Exactly how that is compressed is up to the underlying software (and you can pass through what ever cl args you want).

@tacaswell tacaswell closed this Aug 13, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment