Good visual quality in sdl2 consumer and bad visual quality in avformat consumer

[myme5261314]

This tracker is for defects only! For feature requests, you can submit a pull
request with the changes.

If you’re reporting a defect, make it as detailed as possible, and include both your operating system and MLT versions (e.g. Windows10; MLT v7.25.0).

Windows10; MLT v7.25.0

I encounter a simple use case which is just like below melt command:

melt background.jpg out=":10.000000" width=1280 height=720 -filter rescale \
-track video.mp4 length=":10.000000" -filter shape resource="mask/%d.png" out=":10.000000" use_mix=0 use_luminance=1 \
-filter affine transition.rect="452/-64:471x838" transition.distort=1 \
-transition frei0r.composition out=":10.000000" \
-consumer sdl2 resolution=1280x720 width=1280 height=720 display_aspect_num=1280 display_aspect_den=720 sample_aspect_num=1 sample_aspect_den=1

video.mp4 is with 1080x1920 (portrait screen suitable for mobile phone).

The thing is visual content is clear, including the video-in-video compared to the original video.mp4.
But when I need to save it to output.mp4 it shows with obviously poor visual quality especially for the video-in-video compared to the original video.mp4.

same with the previous command......
-consumer avformat:output1.mp4 \
width=1280 height=720 display_aspect_num=1280 display_aspect_den=720 sample_aspect_num=1 sample_aspect_den=1

I tried it using ffmpeg command like below:

ffmpeg -i background.jpg -i video.mp4 -filter_complex "[1:v]scale=-1:1080[ovr];[0:v][ovr]overlay=500:10" -b:v 1500K -r 25 output2.mp4

The output2.mp4 has similar quality compared to sdl2 preview, which is also better than output1.mp4

So I attempt with vb property.

same with the previous command......
-consumer avformat:output3.mp4 vb=1500K \
width=1280 height=720 display_aspect_num=1280 display_aspect_den=720 sample_aspect_num=1 sample_aspect_den=1

The output3.mp4's file size is greater than output2.mp4, but with almost no difference in visual quality.

I assume sdl2 render the visual quality well, just like image show for each frame, maybe the issue comes along the encoding process of avformat?

I've also checked the png image sequence output, the png file is still in poor visual quality.

same with the previous command......
-consumer avformat:frames/%04d.png vcodec=png vb=1500K \
width=1280 height=720 display_aspect_num=1280 display_aspect_den=720 sample_aspect_num=1 sample_aspect_den=1

Any idea how to investigate more? Can I save each output frame (in ram) to lossless image file to verify my guess?

[bmatherly]

I don't know the specific problem causing the quality loss. But I can offer some advice that might help:

• Consider specifying a profile . You probably do not have to make a new profile because there is probably a built-in profile that will meet your needs. use "melt -query profiles" to see them all.
• Remove all this "width=1280 height=720". Melt will detect the image dimensions.
• Remove "-filter rescale". That is a normalizer filter that is added automatically when needed.
• Remove all this: "resolution=1280x720 width=1280 height=720 display_aspect_num=1280 display_aspect_den=720 sample_aspect_num=1 sample_aspect_den=1" That is all driven by the profile and attempting to override it can result in quality loss (unless you really know what you are doing).

[myme5261314]

After some investigation, I summarized the differences I encountered as below:

1. rescale method matters, I use bicubic instead of default.
2. filter shape makes lossy compression during the frame processing (due to hard-coded yuv422 pixel format

   mlt/src/modules/plus/filter_shape.c

   Line 208 in 70b3ce1

   *format = mlt_image_yuv422;

   )

After fixing these two (the second one requires recompile), the visual effects doesn't seem to be significant blur.

By the way, I notice composite transition seems to have very different visual effects between x-axis coordinate (due to chroma blending I think? )

mlt/src/modules/core/transition_composite.c

Line 348 in 70b3ce1

int uneven_x_src = (x_src % 2);

What I meant to describe is that if you use composite transition to paste an object, whether gemetery x-axis coordinate is odd or even matters, even one result in normal visual effects.
Here are simple commands reproduce the difference.

melt -track color:blue out=100 -track -qtext "+Hello world.txt" out=100 -transition composite geometry="0/1:300x300" out=100 -consumer avformat:hellow1.mp4

melt -track color:blue out=100 -track -qtext "+Hello world.txt" out=100 -transition composite geometry="1/1:300x300" out=100 -consumer avformat:hellow2.mp4

See visual difference between 2 result videos.

hellow2.mp4

hellow1.mp4