Skip to content

VideoMetricsLab v2.0

Latest
Latest

Choose a tag to compare

View all tags
@4KVCD 4KVCD released this 09 Oct 20:30
· 30 commits to master since this release
v2.0
b55ff94

What's new in v2.0

v2.0 is a major performance update. VMAF, VMAF NEG and VMAF v1 now run on
any GPU, the other metrics and Video Compare are several times faster, and the
app uses far less memory.

Metric performance

  • Added Vulkan support for VMAF v1, so it now runs on any GPU (NVIDIA, AMD or
    Intel), up to 17x faster than on a 24 core CPU.
  • Added Vulkan support for VMAF v0.6.1 and VMAF NEG on AMD and Intel GPUs,
    2.3x-2.7x faster than on a 24 core CPU.
  • PSNR, SSIM and XPSNR are now calculated in the app with libvmaf-fast,
    2.7x-11x faster than with FFmpeg.
  • Added software decoding in the app for videos the GPU can't decode, such as
    VVC, making VMAF 1.8x-2.7x faster for them.
  • SSIMULACRA2 and Butteraugli on the CPU are now 7x faster.

System performance

  • Video Compare now plays 4K HDR at 120 frames a second instead of 14, using a
    fifteenth of the CPU.
  • Greatly reduced memory use: the app now idles at 180 MB instead of 840 MB,
    and VMAF v1 at 4K uses 2.6 GB instead of 13.8 GB.
  • Metric Graphs, Video Compare and Settings now open instantly, and the window
    stays responsive with hundreds of videos.
  • Video Compare no longer uses any CPU while idle.
  • Detected black bars are now remembered between sessions.

New features

  • Added a command line, VideoMetricsLab-cli, for running comparisons from
    scripts.
  • Video Compare can now play HDR video in Display P3 colours on the GPU.
  • Added the ability to zoom in Video Compare.
  • Added an option to highlight each metric's best and worst results in the
    Videos tab.
  • Added a dark theme, which follows Windows or can be chosen in Settings.
  • Added drag and drop for videos, pasting a reference path, reordering rows by
    dragging, and sorting by any column.
  • File dialogs now open in the folder your videos last came from, and the app
    can reopen your last session's videos.

Other changes

Bug fixes

  • Fixed XPSNR being weighted by the test video's detail and motion instead of
    the reference's, as XPSNR is defined: scores are 0.9-1.8 dB higher, and
    XPSNR saved by v1.4 is recalculated.
  • Fixed wrong scores, or a run that never ended, for 10-bit H.264 decoded on
    AMD GPUs.
  • CVVDP is no longer calculated on Intel GPUs above 1080p, where it hangs the
    GPU and blacks out the screen.
  • Fixed partly green frames in Video Compare on some NVIDIA GPUs.
  • Fixed the reference showing a few frames behind the test video after a seek,
    for some MP4 references.
  • Fixed FFmpeg running on after the app crashed or was ended in Task Manager.
  • Fixed SSIMULACRA2 and Butteraugli starting again on the CPU, for hours, for a
    test video cut short.
  • Fixed GPU memory growing each time Video Compare moved between an HDR and an
    SDR display.
  • Fixed Video Compare staying blank for a test video added again.
  • Fixed 10-bit H.264 on Intel and AMD GPUs failing first with FFmpeg's GPU
    decoding.
  • A raw .hevc or .264 stream is now refused with how to fix it, not as
    variable-frame-rate video, in Video Compare too.
  • Fixed Video Compare sometimes playing without sound, or with a test video's
    sound instead of the reference's.
  • Fixed choppy sound and slow playback in Video Compare when decoding couldn't
    keep up.
  • Fixed the window freezing on a seek while a video's file was slow to read.
  • Fixed a pause in Video Compare playback when black-bar detection finished.
  • Fixed Video Compare saying it was playing or buffering while paused.

How GPU VMAF works

v1.4 calculated VMAF and VMAF NEG on NVIDIA GPUs only, with its own build of
libvmaf's CUDA code. v2.0 moves that work into
libvmaf-fast, an open-source fork of
Netflix's libvmaf, which adds a Vulkan engine for every GPU:

  • VMAF and VMAF NEG on AMD and Intel GPUs. libvmaf's CUDA code for VIF, ADM
    and motion is ported to Vulkan compute shaders. GPUs round floating point
    differently and some have no double precision, so the shaders use exact
    integer arithmetic and tables made on the CPU instead: every feature equals
    CUDA's bit for bit. NEG reuses everything VMAF calculates except its gain
    limit.
  • VMAF v1 on any GPU. libvmaf has no GPU code for VMAF v1's features (ADM3,
    motion3, CAMBI and SpEED), so the engine follows its CPU code, and every
    score equals the CPU's exactly.
  • PSNR, SSIM and XPSNR are calculated by libvmaf-fast inside the app
    instead of by FFmpeg's filters, with the same scores (XPSNR is FFmpeg's
    filter, ported).
  • NVIDIA GPUs keep libvmaf's CUDA code for VMAF and NEG, now with 13
    upstream pull requests merged, and the fork's own fixes: 4K VMAF + NEG in
    35% less GPU memory.

Where possible, frames from the GPU's own decoder are handed over without
leaving the GPU. Each GPU is checked against known results before it is used,
and one that gets them wrong (a driver bug) is left to the CPU. As in v1.4, the
GPU code runs in a process of its own, so a crash in it or in the driver
doesn't close the app.

Checked on an RTX 5090, the Intel iGPU of a Core Ultra 9 285K and a Radeon
8060S: VMAF and NEG on Vulkan equal CUDA's in 45 cases, and VMAF v1 on the GPU
equals libvmaf's CPU code in 73 cases, at 8 and 10 bits, up to 8K.

Measured

RTX 5090 and Core Ultra 9 285K (its iGPU for the Intel figures, the NVIDIA
GPU hidden); v1.4 against v2.0, whole runs in the app, frames per second:

4K 1080p
VMAF v1 (GPU) 34 → 584 (17x) 102 → 604 (5.9x)
VMAF + NEG (Intel iGPU) 23 → 62 (2.7x) 56 → 126 (2.3x)
PSNR + SSIM + XPSNR 21 → 232 (11x) 67 → 185 (2.7x)
VMAF + NEG, VVC encode 48 → 85 (1.8x) 127 → 341 (2.7x)
SSIMULACRA2 + Butteraugli (CPU) 0.2 → 1.6 (7x) 0.9 → 6.2 (7x)
Video Compare, 4K HDR 120 fps, three videos 14 → 120

Scores match v1.4 frame for frame, except XPSNR (now weighted by the
reference) and VMAF of 1080p encodes, now scaled on the GPU (within 0.16).

Requirements

  • Windows 10 or 11 (x64).
  • FFmpeg 9 or newer with libvmaf, installed separately (not bundled).
  • GPU metrics: a recent NVIDIA, AMD or Intel graphics driver.

Tested on

  • NVIDIA GeForce RTX 5090 (driver 616.92) with Intel Core Ultra 9 285K and its
    integrated GPU.
  • AMD Radeon 8060S (Ryzen AI Max+ 395, driver 32.0.12079.1014).

Download

VideoMetricsLab-windows.zip, SHA-256 277783e73fa6ba4ffedb0412918507a3d38b377fca6606c6611101985e66e30a.

Assets 4
Loading