FFmpeg filter libvmaf use too much memory

[fanyingming]

I use FFmpeg filter libvmaf like:
ffmpeg -i 1.mp4 -i 2.mp4 -lavfi "libvmaf" -f null -
I noticed that the memory FFmpeg used is almost 2x of YUV.
While run_vmaf and ffmpeg2vmaf no such issue.

[LigH-de]

Duplicate of #143