Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize visual quality of converted ugoira #1198

Merged
merged 6 commits into from
Nov 24, 2022
Merged

Conversation

NewUserHa
Copy link
Contributor

@NewUserHa NewUserHa commented Nov 21, 2022

The current ffmpeg paras for converting have some issues:
E.g. apng is a lossless format only, but the current default setting added a denoiser that decreased visual quality and cost more time and increased file size.
E.g. the -r (Set frame rate, can apply to input or output) option is useless and incorrect. because the https://trac.ffmpeg.org/wiki/Concatenate#demuxer creates a variable frame rate input. so this option should never use in most time.

Test used ugoira: https://www.pixiv.net/en/artworks/46281014

Changes:
the .webp now uses lossy encoding with only the least quality loss. the quality is only behind the lossless one.
the .webm (uses vp9) now uses a better bitrate control method, than ffmpeg's default Average Bitrate with a very low 256k.

v20221029 settings:
.png
size: 22,006KB, cost time: 3s
.webm
size: 258KB, cost time: 9s
.webp
size: 1,820KB, cost time: 5s

New settings:
.png
size: 21,667KB, cost time: 2s
.webm
size: 910KB, cost time: 9s
.webp
size: 3,809KB, cost time: 7s

New settings but -lossless 1:
.png
size: 21,667KB, cost time: 2s
.webm
size: 7,045KB, cost time: 10s
.webp
size: 10,420KB, cost time: 10s

The source .zip size: 3,532KB
The original .ugoria size: 3,534KB

@Nandaka
Copy link
Owner

Nandaka commented Nov 22, 2022

You still need to set the -r 999, else the max frame rate will be limited to 25fps.

I'm using https://www.pixiv.net/en/artworks/102233145 as the base and changed the delay manually and the new params failed to create the mp4 with variable frame rate.

modified ugoira
result old param
result new param

{"src":"https://i.pximg.net/img-zip-ugoira/img/2022/10/26/00/00/56/102233145_ugoira1920x1080.zip","originalSrc":"https://i.pximg.net/img-zip-ugoira/img/2022/10/26/00/00/56/102233145_ugoira1920x1080.zip","mime_type":"image/jpeg","frames":[{"file":"000000.jpg","delay":1},{"file":"000001.jpg","delay":1},{"file":"000002.jpg","delay":12},{"file":"000003.jpg","delay":12},{"file":"000004.jpg","delay":1},{"file":"000005.jpg","delay":1},{"file":"000006.jpg","delay":12},{"file":"000007.jpg","delay":12},{"file":"000008.jpg","delay":1},{"file":"000009.jpg","delay":1},{"file":"000010.jpg","delay":12},{"file":"000011.jpg","delay":12},{"file":"000012.jpg","delay":1},{"file":"000013.jpg","delay":1},{"file":"000014.jpg","delay":12},{"file":"000015.jpg","delay":12},{"file":"000016.jpg","delay":1},{"file":"000017.jpg","delay":1},{"file":"000018.jpg","delay":12},{"file":"000019.jpg","delay":12},{"file":"000020.jpg","delay":1},{"file":"000021.jpg","delay":1},{"file":"000022.jpg","delay":12},{"file":"000023.jpg","delay":12}],"zipSize":3008946}
[FFmpeg]
ffmpeg = .\ffmpeg.exe
ffmpegCodec = libvpx-vp9
ffmpegExt = mp4
ffmpegParam = -row-mt 1 -deadline good -crf 20 -vsync 2 -r 999 -pix_fmt yuv420p
webpCodec = libwebp
webpParam = -row-mt 1 -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999
gifParam = -filter_complex [0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle
apngParam = -vsync 2 -r 999 -plays 0
verboseOutput = False

@Nandaka
Copy link
Owner

Nandaka commented Nov 22, 2022

apng default still need to be changed, because looks like got issue with new ffmpeg

@NewUserHa
Copy link
Contributor Author

NewUserHa commented Nov 22, 2022

yes, there's an issue when the duration between images is like 1 ms (which is a rare case though).

the max frame rate will be limited to 25fps.

no. The output plays at the speed that the way you modified. since you can see it play very faster than the original.
The output webm is not vfr, which may be that webm doesn't support vfr probably. but if you change it to mp4 container, you can see the vfr flag in its metadata.

But the real issue I found is that without '-r' it will drop frames. More details can see:
https://superuser.com/questions/908295/ffmpeg-libx264-how-to-specify-a-variable-frame-rate-but-with-a-maximum

To test use:

with Image.open(r"") as im:
  print(im.n_frames)

The source file has total 24 frames. but in the outputs some frames are dropped:
without '-r', the .webm, .webp, .png each have 6 frames, while with'-r' they have each have correct 24+1 frames (+1 is a fix for ffmpeg quirk).
I found, however, the gif is only 10 frames already from old versions.

So I think it is better to completely solve that with a new method, which is '-vsync passthrough'. But there will be some friendly warning info from ffmpeg though.

@biggestsonicfan
Copy link
Contributor

At some point we should look into implementing gifski as an option for ugoira conversion too.

@NewUserHa
Copy link
Contributor Author

NewUserHa commented Nov 22, 2022

gif is a lossy format and an old (low compression)format.

Because the source file is a zip of a sequence of .jpgs, I may choose to use another way to view instead of using any converting.

@Nandaka
Copy link
Owner

Nandaka commented Nov 23, 2022

should we change the -vsync 0 to -fps_mode passthrough, then? Considering it is going to be deprecated in the future?

From: https://ffmpeg.org/ffmpeg.html

-vsync parameter (global)
-fps_mode[:stream_specifier] parameter (output,per-stream)

    Set video sync method / framerate mode. vsync is applied to all output video streams but can be overridden for a stream by setting fps_mode. vsync is deprecated and will be removed in the future.

    For compatibility reasons some of the values for vsync can be specified as numbers (shown in parentheses in the following table). 

as for gif encoding, I don't want to support it directly. You can run post-processing cmd from config.ini by setting enablePostProcessing = True and configure the cmd in postProcessingCmd

@NewUserHa
Copy link
Contributor Author

NewUserHa commented Nov 23, 2022

should we change the -vsync 0 to -fps_mode passthrough?

ffmpeg 5.0 has no -fps_mode, even it is a pretty new version and released in recent months.
I don't think ffmpeg will remove -vsync any soon. So using -vsync should be better.

apng default still need to be changed, because looks like got issue with new ffmpeg

Does the apng default now have any issue?

@Nandaka
Copy link
Owner

Nandaka commented Nov 23, 2022

looks ok with -plays 0 -vsync 0

@NewUserHa
Copy link
Contributor Author

Then this PR is complete.

@Nandaka Nandaka merged commit f10f8cb into Nandaka:master Nov 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants