Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run VMAF Library in FFMPEG #111

Open
ecomte opened this issue Aug 28, 2017 · 41 comments

Comments

Projects
None yet
@ecomte
Copy link

commented Aug 28, 2017

After reading documentation, I saw it is able to run VMAF via a plugin on ffmpeg. I have yet to figure out how to though.

ffmpeg says "simply" install libvmaf, and run ./configure --enable-libvmaf. This makes sense, but there are no ./configure files to be found and after scrounging everywhere to figure out how this is possible, I have come up empty. I have significant experience using VMAF outside of ffmpeg, but I would very much like to run VMAF through the libvmaf on ffmpeg. Please let me know how this is possible.

Thanks

@li-zhi

This comment has been minimized.

Copy link
Contributor

commented Aug 28, 2017

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

Hi @ecomte , configure file is there inside FFmpeg/ so if you want to run vmaf using FFmpeg first install libvmaf library using make install and then from the root directory of FFmpeg (which is FFmpeg/) do ./configure --enable-libvmaf . After that do make install. Run vmaf through FFmpeg using the following command:
ffmpeg -i main -i ref -lavfi libvmaf -f null -
Also, libvmaf was changed recently so I updated the patch on FFmpeg which is not yet applied so you may get some warnings and errors on installing and running libvmaf. These two patches are yet to be applied: https://patchwork.ffmpeg.org/patch/4610/ and https://patchwork.ffmpeg.org/patch/4575/.
We will update soon.
You can follow this updated file on my personal repo:
https://github.com/ashk43712/FFmpeg/blob/libvmaf/libavfilter/vf_libvmaf.c
Let me know if you still face some issues.
Thanks.

@ecomte

This comment has been minimized.

Copy link
Author

commented Aug 30, 2017

Hi. Thank you for your quick response, I greatly appreciate it. Will I have to do make install libvmaf from an external source, or is the library included in the FFmpeg development package?

Thanks.

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2017

Hi @ecomte, FFmpeg only links to libvmaf, the library itself is not included in the FFmpeg development package. You can install libvmaf from Netflix's VMAF package (this repo) using make and then make install.

@ecomte

This comment has been minimized.

Copy link
Author

commented Aug 30, 2017

Hi @ashk43712. I was able to successfully install libvmaf. After installing, I "cd'ed" into the root FFmpeg directory and ran "./configure --enable-libvmaf". It gave the following error: "ERROR: libvmaf not found using pkg-config". Do you happen to have any idea what this could be? I was wondering if it could be an issue with how I have my CentOS machine set up, or if it is a problem with FFmpeg and the newly installed libvmaf.

Thanks.

@chuanY

This comment has been minimized.

Copy link

commented Aug 31, 2017

@i have same issues too.
hopes you can fixed.

and my vmaf version is 1.3.1

thanks

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2017

Hi @chuanY , can you mention your issues?

@chuanY

This comment has been minimized.

Copy link

commented Sep 1, 2017

when i run "./configure --enable-libvmaf" , i get "ERROR: libvmaf not found using pkg-config" issues

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Sep 1, 2017

Hi, @ecomte and @chuanY . Can you check if the pkg-config file (libvmaf.pc) is copied inside /usr/local/lib/pkgconfig/ after running make install. Can you also check if the library (libvmaf.a) and the
library header (libvmaf.h) are copied to usr/local/lib/ and usr/local/include/ respectively. If these are not their then mostly the issue is with your system, maybe permissions , not sure though. Can you also try running them using sudo (sudo make install). FFmpeg should detect these files if it's there.
Thanks

@ecomte

This comment has been minimized.

Copy link
Author

commented Sep 1, 2017

I checked all of those directories and they are containing the correct files. I tried running sudo make install, and then './configure --enable-libvmaf' and still got the same results: "ERROR: libvmaf not found using pkg-config".

@chuanY, let me know what you find out, I'm curious to see. Maybe it is an issue with my machine, but I will wait for an update to see if you're still having the same issues.
Thanks

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2017

Hi @ecomte and @chuanY, can you check the default path of your system's pkg-config files (using "pkg-config --variable pc_path pkg-config" or similar commands). Can you try moving libvmaf.pc at one of these paths and see if it works.
I think it makes sense to make the install location customizable to fix system specific issues.
Thanks.

@chuanY

This comment has been minimized.

Copy link

commented Sep 5, 2017

hi @ashk43712 @ecomte i fixed by i use latest master branch code, not 1.3.1 release tag, and i success

@nandakd

This comment has been minimized.

Copy link

commented Sep 6, 2017

Was able to configure ffmpeg with vmaf. Check ffbuild/config.log - you may need to set PKG_CONFIG_PATH.

@slhck

This comment has been minimized.

Copy link
Contributor

commented Sep 25, 2017

So, just to summarize, for VMAF:

  1. Clone https://github.com/Netflix/vmaf
  2. Run the installation according to https://github.com/Netflix/vmaf/blob/master/resource/doc/libvmaf.md#use-libvmaf-with-ffmpeg

For FFmpeg:

  1. Clone https://git.ffmpeg.org/ffmpeg.git
  2. Install the prerequisites (see the Compilation Guides)
  3. Run ./configure --enable-libvmaf (including other configuration switches if needed)
  4. Run make, and optionally make install
@nandakd

This comment has been minimized.

Copy link

commented Sep 29, 2017

Has anyone tried executing the vmaf command itself through ffmpeg?

I've been trying based on the description https://ffmpeg.org/ffmpeg-filters.html#libvmaf, but not successful so far.

@slhck

This comment has been minimized.

Copy link
Contributor

commented Sep 29, 2017

@nandakd If you're having issues, you need to show exactly what you've been doing and what output / errors you are getting. When you say that you were "not successful", that's not really a proper problem description.

@nandakd

This comment has been minimized.

Copy link

commented Sep 29, 2017

I tried the exact examples mentioned on that ffmpeg page, and got an ffmpeg segmentation fault.

I then tried using ffmpeg2vmaf, which worked, but does not support specifying a log file - need to use stdout/stderr redirecting . ffmpeg2vmaf also exits with this weird tty error, where echo is turned off (needs a reset).

@slhck

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2017

got an ffmpeg segmentation fault

Same here – I've reported it on the FFmpeg bug tracker

@ashk43712

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2017

Hi everyone, I have started looking into the issues. I'll try to resolve them as soon as possible.

@rbultje

This comment has been minimized.

Copy link

commented Nov 6, 2017

The segfault from ffmpeg bug #6691 is now fixed since 4c38aa79ece4fa0ea7533774197d6408d7d3b9e5.

@GyanD

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2017

I've compiled libvmaf on Windows for integration with FFmpeg, so to clarify on what @slhck said above , python and its packages are not required in order to compile either the static libvmaf.a library or the standalone vmafossexec tool.

@xshen1122

This comment has been minimized.

Copy link

commented Dec 8, 2017

@ashk43712
I have successfully run ffmpeg + libvmaf for mpg file like: ffmpeg -i main.mpg -i ref.mpg -lavfi libvmaf -f null -
But I want to compare yuv file, "ffmpeg -i 1.yuv -i 2.yuv -lavfi libvmaf -f null - " , ffmpeg cannot regonize yuv:
[IMGUTILS @ 0x7ffc653935c0] Picture size 0x0 is invalid
1.yuv: Invalid argument

Any idea? How to compare 2 yuv files?

@GyanD

This comment has been minimized.

Copy link
Contributor

commented Dec 8, 2017

That's an ffmpeg issue, not VMAF. The short answer is that since the inputs are raw, there's no metadata to inform ffmpeg about the properties of the input. ffmpeg needs that to initialize the decoder. For raw YUV, you'll have to specify the resolution, frame rate and pixel format for each input e.g. -video_size 1280x720 -pixel_format yuv420p -framerate 30 -i 1.yuv

@xshen1122

This comment has been minimized.

Copy link

commented Dec 8, 2017

@GyanD 👍
Thanks a lot. It's working now.

@chuanY

This comment has been minimized.

Copy link

commented Dec 12, 2017

still get segmentation fault at centos run ffmpeg @rbultje

@rbultje

This comment has been minimized.

Copy link

commented Dec 12, 2017

The patch isn't in ffmpeg yet, so you're probably running the old version.

@chuanY

This comment has been minimized.

Copy link

commented Jan 8, 2018

@rbultje what's time patch can be include as ffmpeg main branch

@rbultje

This comment has been minimized.

@soulhdb

This comment has been minimized.

Copy link

commented May 4, 2018

my server has 16 cores,but when run libvmaf(do vmaf or psnr or ssim) filter only use one,is there a problem?

#131
and just as the issues describe, libavfilter/vf_libvmaf.c line:320 “pthread_join(s->vmaf_thread, NULL);“ cause segment fault

CentOS + ffmpeg4.0 + glibc2.17

@KevinJW

This comment has been minimized.

Copy link

commented May 4, 2018

http://ffmpeg.org/pipermail/ffmpeg-devel/2018-April/229205.html contains a suggested fix for the pthread_join() issue in ffmpeg.

Kevin

@GyanD

This comment has been minimized.

Copy link
Contributor

commented May 4, 2018

Above patch available at https://patchwork.ffmpeg.org/patch/8699/

@MukeshA1

This comment has been minimized.

Copy link

commented Jun 15, 2018

Hi,
Is VMAF library now bundled with ffmpeg or do we still need to build it separately? I am trying to use VMAF on windows OS. Is there a way I can directly download the executable/library and integrate with ffmpeg on windows ( in case it is not bundled with ffmpeg)?
Thanks!

@GyanD

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2018

FFmpeg contains a wrapper for VMAF, but the library is separate and has to be linked. Pass --enable-libvmaf to ffmpeg's configure.

However, at the present time, I believe ffmpeg build with libvmaf HEAD will fail as the VMAF API has changed and the wrapper hasn't been updated.

@MukeshA1

This comment has been minimized.

Copy link

commented Jun 15, 2018

Hi Gyan,

At https://github.com/Netflix/vmaf , I see the instructions for getting the vmaf only for linux and mac os. Any idea what I need to do to get the vmaf tool on Windows. Is there any repository where the compiled version of the tool is present for Windows OS? ( As no instructions are mentioned for Windows OS, I am just worried whether there will be quite a few prerequisites for building it on windows..)

Thanks

@slhck

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2018

There is no pre-built version that I know of. Compiling ffmpeg under Windows is a little more complicated.

You may be able to use https://github.com/jb-alvarado/media-autobuild_suite which should enable libvmaf.

@li-zhi

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2018

@GyanD @MukeshA1 release v1.3.5 or v1.3.4 should still work with current FFmpeg build.

@cosmin

This comment has been minimized.

Copy link

commented Aug 22, 2018

Latest snapshot of ffmpeg (as of 8/7 I believe) will compile against latest libvmaf (1.3.9). I think this issue can be closed.

@yayaMissie

This comment has been minimized.

Copy link

commented Mar 19, 2019

So, just to summarize, for VMAF:

  1. Clone https://github.com/Netflix/vmaf
  2. Install the prerequisites and the required pip commands
  3. Run make, edit your PYTHONPATH, and run make install

For FFmpeg:

  1. Clone https://git.ffmpeg.org/ffmpeg.git
  2. Install the prerequisites (see the Compilation Guides)
  3. Run ./configure --enable-libvmaf (including other configuration switches if needed)
  4. Run make, and optionally make install

my question is:

  1. How to edit PYTHONPATH and Why should we do the step?
  2. I have followed the steps mentioned above(except doing editing PYTHONPATH and not using make install cmd). When using libvmaf, ffmpeg throw exception as:
    ~/ffmpeg$ ./ffmpeg -i /mnt/ssd/videos_qaperf/videos_input/qaperf_30fps_1min_720p_2M_264_gop10.mp4 -i /mnt/ssd/videos_qaperf/videos_input/qaperf_30fps_1min_720p_2M_264_gop10.mp4 -libvmaf -f null - ffmpeg version N-93397-gf8075b2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 4.9.2 (Debian 4.9.2-10+deb8u1) configuration: --enable-libvmaf --enable-version3 libavutil 56. 26.100 / 56. 26.100 libavcodec 58. 47.103 / 58. 47.103 libavformat 58. 26.101 / 58. 26.101 libavdevice 58. 6.101 / 58. 6.101 libavfilter 7. 48.100 / 7. 48.100 libswscale 5. 4.100 / 5. 4.100 libswresample 3. 4.100 / 3. 4.100 Unrecognized option 'libvmaf'. Error splitting the argument list: Option not found
    Why does this error happen? How can I fix the problem?
@GyanD

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2019

libvmaf is used via a filter, so it's -vf libvmaf.

@slhck

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2019

How to edit PYTHONPATH

Edit your shell configuration file (e.g., ~/.bashrc), see here: https://bic-berkeley.github.io/psych-214-fall-2016/using_pythonpath.html

Why should we do the step?

It's actually just required for running the Python wrapper for VMAF since the program is not installed as a package. For running the libvmaf filter in ffmpeg, this step should not be necessary. (I'll edit my comment above to reflect that.)

The whole procedure is documented here: https://github.com/Netflix/vmaf/blob/master/resource/doc/libvmaf.md#use-libvmaf-with-ffmpeg

@yayaMissie

This comment has been minimized.

Copy link

commented Apr 10, 2019

Thanks. @slhck @GyanD
There are some problems when using libvmaf via ffmpeg.

  1. libvmaf is used as a filter, but "-vf libvmaf" is simple filter graph, which is only for one input and output. So I changed to "-filter_complex/-lavfi"
  2. The cmd I used is as follows:
    ./ffmpeg -i input.mp4 -i ref.mp4 -filter_complex "[0:v][1:v]libvmaf" -f null -
    But it doesn't work. The following shows the error I encountered:

Stream mapping:
Stream #0:0 (h264) -> libvmaf:main (graph 0)
Stream #1:0 (h264) -> libvmaf:reference (graph 0)
libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Start calculating VMAF score...
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.27.100
Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default)
Metadata:
encoder : Lavc58.48.101 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc58.48.101 pcm_s16le
Caught runtime_error: StatVector size is 0.
[Parsed_libvmaf_0 @ 0x3f445c0] libvmaf encountered an error, check log for details
Error while filtering: Invalid argument
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #1:0
Conversion failed!
Start calculating VMAF score...
Caught runtime_error: StatVector size is 0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.