FFmpeg Integration - mp4, webm, gif #624
Conversation
|
You are awesome! Thank you so much :D |
|
gif import would be amazing too - but its probably a lot of work :) Still great progress on the ffmpeg front!! We finally have internet friendly output - out of the box |
|
Added an intermediate conversion to jpg to delete the raw image. Without the jpg coversion, a 5 minute 24fps animation would take 57gb in the process. It would be deleted after the final output, but a failed conversion would eat hard drive space. With the jpeg conversion, it drops to around 1.4gb. That is still freed after the final output, but is a much better figure to be dealing with. |
|
@blurymind If I made a portable version with this feature, would you test it out and try and break it? |
|
@turtleTooth of course i will test it. :) if it helps getting it merged |
|
Jpeg will give compression artifacts, especially for solid color. Why not use PNG as an interchange format? (compresses solid color well). People who export long animations probably would export directly as image files anyway (to allow for partial re-rendering). |
|
@turtleTooth its also worth noting that webm supports transparency: |
|
There is no other video format that i know of that also supports having an alpha channel :b its pretty cool. If you use png instead of jpg - you might be able to export the alpha channel too |
|
Google claims that webp has much better quality and compression than png. That is another format worth investigating. There are other formats too - but webp is solid as it is supported by google and already used by the web |
|
PNG is lossless, so not sure what better quality means in that context. |
|
Note that its possible to avoid writing files at all, which avoids many files on disk and time to compress as one format then re-encode into another. See:
Its possible to set this up for the user without them having to manually run external commends (as is done already for this PR). |
|
Ah sorry - i totally overlooked that point. I guess only advantage might be file size. Note that webp supports both lossless and lossy |
|
@ideasman42 that does seem better. @turtleTooth was earlier asking for help on the ffmpeg exporter. Blender does use ffmpeg as well and it works great there |
|
Aligorith wrote some notes on export with it a couple of years ago http://aligorith.blogspot.co.uk/2014/01/blender-ffmpeg-notes-on-getting-usable.html?m=1 |
|
@ideasman42 I chose jpg for file size, but I did choose the highest quality. You bring up a good point about compression artifacts. I will try it with png and compare file sizes and see how it turns out. As for a frameserver, that is probably the best long term solution. However, it looks like it involves creating a network or socket between one program and the other. I have never done anything with networking before and don't feel equipped to take that on just now. It seems like it would be very doable as OT serves up raw frames and I am just saving those. It would be a much more efficient solution. Right now mine saves the raw frame, converts it to jpg (or png), deletes the raw frame to save disk space, converts all jpg (or png) frames to video or gif, and then deletes the jpg files. There are definitely intermediate steps that would be great to get rid of. Do you have any experience writing a frameserver? Is that something you might be able to help out with? BTW- thanks for all your great contributions so far to the program. Your technical knowledge is a great asset. Update: png looks good and the filesize is good. |
|
@blurymind, Blender's ffmpeg is built-in, so it wont help with integration in this case (the frameserver can be used with ffmpeg, but its not ffmpeg spesific). @turtleTooth, I didn't write Blender's frame-server, but reading over the code its quite simple, I've tested using it, it works quite well. Regrettably I wont be able to help with this task, just giving some hints on alternative ways to do this. |
|
Thanks for the feedback. It sounds like a good goal to add in the future. In the meantime, this solution seems to work. It is just a little slow. |
|
As far as WebP goes, it seems that at least the Mac version of OpenToonz is deployed with the WebP QT plugin dll, so it may be possible to add support through |
|
@yamisama well spotted. It might be advantageous to preserving space . So 1 gb coud go down to 500 or even 250 mb! Which would be a huge win.
in example case - webp is 1/3 of the size of the png. that is 150% smaller! Lets not also forget that webp and webm are practically a brother and a sister. So I would imagine it would work better for webm export. I believe that if using webp brings down the size so much - not having a frameserver for a while will not be noticeably bad. |
|
@blurymind I will test it out. It looks very promising and has the added benefit of making webp export already baked in. OT already supports png out, so using webp as the intermediate adds webp as an export format too. |
|
@yamisama Thank you so much for letting me know about the built in webp. I wasn't able to use it, because the rendering time was too slow (but the file sizes were amazing), but it made me look up saving a QImage which is built into Qt. I'm still learning Qt, but this allowed me to bypass the step of saving the raw image, and then converting to an intermediate format. I can save to PNG directly. @blurymind I will probably use this method for creating webp exports, but note that the process is slow, However, the file sizes were 1/3 of a jpg at 100% quality and half of a png. - Amazing and great quality. I think slowness is the tradeoff for the filesize. |
|
Just tested: |
|
The portable version includes ffmpeg and t32bitsrv - you can test it if you want. |
|
Does the homebrew install of ffmpeg include ffprobe? Import requires ffprobe - I added a check for it today. |
|
Yes, ffprobe is bundled with ffmpeg installed with homebrew. Anyway, I will try latest version. Thanks. |
| @@ -159,6 +162,33 @@ void initImageIo(bool lightVersion) { | |||
| TFileType::declare("rgb", TFileType::RASTER_IMAGE); | |||
| Tiio::defineWriterProperties("rgb", new Tiio::SgiWriterProperties()); | |||
|
|
|||
| // ffmpeg | |||
| if (Ffmpeg::checkFfmpeg()) { | |||
There was a problem hiding this comment.
Please add a following line before the line 166:
#if !defined(_WIN32) || defined(x64)
and after the line 189, please add #endif
In Ffmpeg::checkFfmpeg() you are checking the value in Preferences, and Preferences tries to see TOONZPROFILES path from registry (in its ctor). This behavior seems to abort t32bitsrv.exe since the TEnv paths are not set in t32bitsrv.exe's main() function.
Adding #if !defined(_WIN32) || defined(x64) is to avoid this problem by skipping Ffmpeg::checkFfmpeg() on running t32bitsrv.exe. This check must be only needed for OpenToonz_1.0.exe.
|
I added the check above. @shun-iwasawa Great catch! I also think I might have figured out the Mac crash on import. I was parsing the ffprobe output and splitting the string using \r as a delimiter. I don't think Mac uses \r - only \n. So the new line is: |
@Rayek I was able to get the sliders working again and actually keep the values when the settings were reopened. So now there is much more granular control over the scale and quality. Once the initial commit is merged I will probably work on an advanced input dialog where users can input their own settings for ffmpeg to run. This would take knowledge of how ffmpeg takes it's commands, but would give the user more control. |
|
Jenkins, build this |
|
Hoping this works for you. After you test this, I have added the ability to set how long ffmpeg can run before timing out. I will wait to push it until you get a chance to test this. |
|
@turtleTooth I think you don't need to hesitate to push the changes. You can just revert the change if it doesn't work! |
|
Oh sorry, I mean "you don't need to wait to push the changes." |
That works too. |
|
Today's Check OSX
Windows
|
|
|
||
| QString palette; | ||
| QString filters = "fps=" + QString::number(m_frameRate) + ",scale=" + | ||
| QString::number(outLx) + ":-1:flags=lanczos"; |
There was a problem hiding this comment.
I think you should remove the "fps" filter here and use "-r" input option instead.
To be more precise, please modify the lines 74-75 to be as follows;
QString filters = "scale=" + QString::number(outLx) + ":-1:flags=lanczos";
...and insert following lines to the line 93;
preIArgs << "-r";
preIArgs << QString::number(m_frameRate);
When I checked your PR, I found some problem: If you set a scene fps to other than 25, some frames are dropped in the output GIF. For example, when you render a scene with 20 frames, the result GIF becomes 19 frames under 24fps (= the default value of OpenToonz) and 11 frames under 12.5fps.
According to the documentation of ffmpeg filters, it says;
fps
Convert the video to specified constant frame rate by duplicating or dropping frames as necessary.
I'm new to ffmpeg so I'm not 100% convinced, but this option may be the cause of this problem.
On the other hand, the "-r" option is documented here as follows:
-r[:stream_specifier] fps (input/output,per-stream)
Set frame rate (Hz value, fraction or abbreviation).
As an input option, ignore any timestamps stored in the file and instead generate timestamps assuming constant frame rate fps.
Here is a comparison of the rendered results. Both results are rendered from the same scene which has 20 frames in length and with 8 fps frame rate. Please note that some frames are dropped from the former result.
- With "fps" filter (current implementation):
- With "-r" input option (my suggestion):
|
@kogaki @shun-iwasawa Thank you for the help. I added the change and caught an error where scaling wouldn't happen if a palette wasn't generated. Gif fps now works as expected. Great suggestion. |
|
Jenkins |
|
@turtleTooth I think it's time to be merged because basic I/O feature works. Do you think this is still "WIP" PR ? |
ghost
changed the title
|
I totally agree. I'm sure things will come up that will need to get fixed, but it's all basically there. I will squash bugs as they come up. Thank you so much for all your help. |
|
Let's do this! |
|
Merged! Thank you for your great contribution and continuous improvements ! |
|
Thank you! Thank you! |
|
Thank you @turtleTooth ! This is an epic addition On 29 Jul 2016 06:59, "turtletooth" notifications@github.com wrote:
|
ghost
mentioned this pull request
ghost
deleted the
Gif and Webm export are working when 1 cpu is used for export. I modified outputsettingspopup to disable other cpus when gif or webm is selected. Obviously it would be better if it could use all, but it is inconsistent right now.
I submitted these both together because the implementation is basically the same(as is the h264 PR). I can split them if you prefer, but anything that doesn't work on one will probably not work on the other. The only difference is ffmpeg options.
I realize that h264 support is being debated due to licensing issues, but these options shouldn't have licensing issues.
The code works but I tagged it WIP because I'm sure there are things I missed. If you could look at it, I would appreciate it.