FFMpeg can be used for calculating different visual quality metrics (PSNR, SSIM, VMAF). FFMetrics is a FFMpeg GUI that purpose is to visualize quality metrics calculated by FFMpeg. The program allows you to select files without dealing with command line, calculate & visualize PSNR, SSIM & VMAF quality metrics for all of them in one go.
Well, and build shiny interactive graphs of course:
- PSNR, SSIM, VMAF visual quality metrics;
- Processing up to 24 files in one go (this can be increased by configuring graphs' styles) ;
- No limitations on frame size for PSNR/SSIM, Full HD/4K for VMAF;
- Brief media info for reference and distorted files;
- Thumbnail for reference file;
- “Bad” frames can be extracted and saved as PNG images for further analisys;
- Only parts of video files can be analyzed;
- Easy to use UI: drag & drop files from Explorer onto Reference field and Files list or use file choosers;
- Frames graphs can be zoomed in/out with mouse wheel (try it over graph or graph's axes), panned with right mouse button and saved as SVG or PNG;
- Frames graphs are optional and can be turned off in menu or with command line option;
- Frames metrics can be saved as tab-delimited csv files and then opened in Excel;
- FFMpeg commands issued by FFMetrics can be saved to log file (
FFMetrics.log); - Average metrics, frames statistics, frame size, bitrate, date/time and file name can be saved to tab-delimited csv file (appended) and then opened in Excel;
- VMAF model selected automatically based on reference media info, but can be changed using UI;
- Supported VMAF models type (in-build in FFMpeg, pkl or json) detected automatically;
- Most program options could be supplied as command line parameters;
- Free. No registration, banners, tracking etc.
- Windows OS
- .NET Framework 4.8.0 or later. The framework is already included since Windows 10 1903 so you do not need to install it separately. However, if you use earlier versions of Windows 10 or Windows 7/8, the program should ask you to download and install it.
- FFMpeg.exe. You need to download it from official ffmpeg web site. You can try static build for simplicity. However, for real usage I'd recommend to use shared build.
- VMAF metric require special FFMpeg's build. It is supported since FFMpeg version 4.3 (stable). However, you'd better use FFMpeg 4 or newer.
In addition, VMAF model files must be in sub-folder
vmaf-models. The most common models are already included in archive. You can get other models from Netflix VMAF project. If you use the most recent FFMpeg builds, the vmaf models could be in-build into it so you do not need to use separate vmaf models.
- Unpack into a folder;
- Put FFMpeg.exe (and accompanied dll files if you use shared build) into the program folder or make it available through system %PATH%;
- Run the program;
- Use UI to add reference file and at least one distorted file (you can drag & drop files from Explorer or use file choosers);
- Select metrics you'd like to calculate;
- Click “Start” button.
FFMetrics.exe [options] ref.mp4 file1.mp4 [file2.mp4] [file3.mp4] [...]
-auto-save-results After calculation save results to log
-auto-save-results-file=C:\path\file.csv
-duration=<seconds> Duration of video stream to be processed
Default: 0 (whole stream used)
-log-commands Log ffmpeg commands
-log-frames Log frames' metrics in csv files
-log-frames-dir=C:\path\ Folder where frame's metrics will be stored
-log-level=DEBUG|ERROR|INFO Default: INFO
-metric=PSNR|SSIM|VMAF Default: all (-metric=PSNR -metric=SSIM -metric=VMAF)
-plot-update-speed=HIGH|NORMAL|LOW|OFF Plots update frequency
Default: NORMAL
-plot-window-manual Do not open plot window automatically when starting calculation
-project=C:\path\to\project.ffmproj Read project options from specified file
-run Run calculation when program started
-scaling-method=NEIGHBOR|GAUSS|BILINEAR|BICUBIC|LANCZOS|SINC|SPLINE
Default: BICUBIC
-skip=<seconds> Duration of video stream to be skipped
Default: 0 (stream processed from the beginning)
-temp-dir=dirspec Default: default user temporary directory
-vmaf-model=filename Default: detected automatically based on reference media info
-vmaf-phone-model
-vmaf-pool=MEAN|HARMONIC_MEAN Default: MEAN
-vmaf-subsample=<value> 1 means VMAF metric for every frame to be calculated
2 means VMAF metric for every 2nd frame to be calculated
And so on
Default: 1
All options can be provided using single leading dash (-option) or double leading dash (--option).
FFMetrics.exe \\server\path\to\ref.mp4 "c:\path\to\my file.mp4"
FFMetrics.exe -log-frames -metric=SSIM -metric=VMAF -run c:\to\ref.mp4 c:\to1\file1.avi c:\to2\file2.avs
FFMetrics.exe -project=c:\path\to\project.ffmproj -vmaf-model=vmaf_v0.6.1neg -vmaf-pool=harmonic_mean -run
- You have to be very careful and supply video files in the same colour range or with correct colour range's meta. Otherwise ffmpeg.exe could make incorrect transformation and give you incorrect results (more details). I do have plans to improve the program to prevent/minimize this happen.
- Close FFMetrics and delete
FFMetrics.log; - Run the program with option
-log-level=debug; - In program menu activate “Options | Write FFMpeg commands to log” (required for version below 1.3.0);
- Add reference file;
- Add one distorted file and make it active (checkbox on the left of filename must be ticked);
- Make sure at least one metric is active (checkbox on the left of metric name must be ticked);
- Click “Start” button;
- Take screenshot (Alt+PrnScr or Win+Shift+S and paste it into image editor and save as PNG);
- Close the program;
- Analyze
FFMetrics.log. You can try to run the ffmpeg command directly; - Upload archived
FFMetrics.logwith screenshot to dropbox (or similar) and share the link.
- “Start” button disabled
- No FFMpeg.exe found;
- No reference file added, reference file does not exist (red file name) or program is unable to get file's media info;
- No distorted file added, distorted file does not exist (red file name), program unable to get file's media info or no active distorted files (files with checkbox on the left of the file name ticked);
- No metrics selected.
- VMAF checkbox disabled
- FFMpeg.exe does not support VMAF. Download newer version, make sure it supports VMAF.
- No
vmaf-modelsfolder with supported models in it. FFMpeg might support pkl model but there are only json models in the folder or visa versa. - Program path contains non-English characters. FFMetrics itself should not have issues with non-English characters in paths, but FFMpeg.exe could fail while trying to open VMAF model file during FFMpeg feature-detection startup process.
- Error while calculating VMAF metric
- Invalid VMAF model file. The first thing that you should check if you downloaded models on you own. Model file must be less than 30KB and should not contain HTML in it.
- I'm trying to calculate VMAF metric comparing the file with itself and not getting score 100.
- Based on VMAF FAQ this is by design.
fifonik
- Discussion in Vegas forum (in English)
- Discussion in IXBT forum (in Russian)