NVIDIA Vulkan Video VA-API Prototype

This is a clean VA-API driver prototype for NVIDIA GPUs using Vulkan Video. The goal is modern browser decode first: expose a small, codec-aware VA-API driver that translates VA decode requests into Vulkan Video and exports browser-importable dma-bufs.

This is not a fork of the NVDEC-backed driver design. The public boundary stays as libva's C ABI, while the implementation is organized as C++23 modules for VA objects, codec parsing, Vulkan sessions, surface resources, and dma-buf export.

Current State

The driver currently builds and initializes as:

NVIDIA Vulkan Video VA-API prototype 0.1.0

Live state on the current development machine:

Vulkan runtime codecs: h264,hevc,vp9,av1
Vulkan profile caps: h264, h265, h265_10, h265_12, vp9, vp9_10, vp9_12, av1, av1_10
Export caps: NV12 and P010
Advertised VA decode profiles: H.264, HEVC Main, HEVC Main10, VP9 Profile0, VP9 Profile2, AV1 Profile0

vainfo currently advertises:

• VAProfileH264ConstrainedBaseline : VAEntrypointVLD
• VAProfileH264Main : VAEntrypointVLD
• VAProfileH264High : VAEntrypointVLD
• VAProfileHEVCMain : VAEntrypointVLD
• VAProfileHEVCMain10 : VAEntrypointVLD
• VAProfileVP9Profile0 : VAEntrypointVLD
• VAProfileVP9Profile2 : VAEntrypointVLD
• VAProfileAV1Profile0 : VAEntrypointVLD

Important limits:

• HEVC Main and Main10 are wired and advertised when Vulkan Video and export support are present. HEVC Main12/P012 remains hidden because P012 export is not wired.
• AV1 Profile0 now supports both the 8-bit NV12 path and the 10-bit P010 path.
• VP9 Profile2 uses the P010 path. 12-bit/P012 stays hidden because P012 export is not wired.
• Encode entrypoints are deliberately not advertised. The tree has structural hooks for a later encode path, but main does not expose VA-API encode.

What Works

• C++23 VA-API driver module exporting __vaDriverInit_1_0.
• Runtime Vulkan Video probing through libvulkan.
• Runtime codec selection for H.264, HEVC, VP9, and AV1 decode.
• H.264 8-bit NV12 browser decode path.
• HEVC Main 8-bit NV12 decode path.
• HEVC Main10 10-bit P010 decode path.
• VP9 Profile0 8-bit NV12 browser decode path.
• VP9 Profile2 10-bit P010 browser decode path.
• AV1 Profile0 8-bit NV12 browser decode path.
• AV1 Profile0 10-bit P010 decode/export path.
• Single-FD, two-layer dma-buf export for NV12 and P010.
• Export shadow images for NVIDIA's optimal-only decode images.
• Retained export backing logic for Chrome stream switches and imported external surface pools.
• Non-blocking decode completion path with pending work drained on sync/export.
• Smoke tests for codec capability, sessions, export lifecycle, retained export backing, VA lifetime, sync, multi-context, and import handling.

Layout

src/
  caps/                 VA profile, format, and config capability records
  codecs/               VA buffer parsers and codec operation registry
  codecs/{h264,hevc,vp9,av1}
                        Codec-owned VA decode state
  va/                   libva entrypoints, objects, configs, contexts, surfaces
  vulkan/               runtime, command submission, sessions, resources, export
  vulkan/codecs/*       Vulkan Video codec session/decode implementations
tests/smoke/            Focused smoke tests for driver and codec behavior

Build

Required development packages include libva, libva-drm, libdrm, Vulkan headers and loader, GStreamer codecparsers, Meson, Ninja, and a C++23 compiler.

Default optimized build:

make all

Equivalent Meson commands:

meson setup build -Doptimization=2 -Ddebug=false -Db_ndebug=true
meson compile -C build

Experimental local-machine build:

make experimental

The experimental target uses -O3 and -march=native. It is useful for local testing, but the default build avoids machine-specific code generation.

Test

Run the smoke suite:

make test

The Vulkan and VA export tests need access to the real Vulkan/DRM devices. If a sandbox blocks device access, failures commonly look like vkCreateInstance failed or missing /dev/dri/renderD* nodes rather than source failures.

Run the hardware capability probe:

./build/vkvv-probe

Run vainfo against the in-tree driver:

LIBVA_DRIVER_NAME=nvidia_vulkan \
LIBVA_DRIVERS_PATH="$PWD/build" \
VKVV_LOG=1 \
LIBVA_MESSAGING_LEVEL=2 \
vainfo

Install

Build the driver variant you want to install:

make all

or, for a local-machine build:

make experimental

Install the already-built driver into libva's system driver directory:

sudo make install

make install only installs build/nvidia_vulkan_drv_video.so; it does not rebuild or switch optimization modes. Like the original NVIDIA VA-API driver, Meson gets the install directory from libva's pkg-config driverdir variable. Check the resolved directory with:

pkg-config --variable=driverdir libva

For local browser testing without installing:

LIBVA_DRIVER_NAME=nvidia_vulkan \
LIBVA_DRIVERS_PATH="$PWD/build" \
VKVV_LOG=1 \
google-chrome-stable

Development Commands

make format-fix   # clang-format all src/ and tests/ C++ files
make tidy         # run clang-tidy using the Meson compile database
make clean        # clean Meson build outputs

Pending Work

Near-term driver work:

• Keep hardening retained export backing and transition-window behavior so Chrome stream switches stay deterministic without holding excessive VRAM.
• Lower and tune detached/retained export memory caps after more transition testing at 1080p, 4K, NV12, and P010.
• Expand browser transition telemetry only where it catches real stale-frame, grey-frame, or imported-backing regressions.
• Build a repeatable browser sample matrix for H.264, VP9 SDR, VP9 HDR/P010, AV1 SDR/HDR, HEVC Main/Main10, resolution changes, and codec switches.

Codec work:

• Complete VP9 hardening: show-existing frames, superframes, invisible references, alt-ref behavior, resolution/profile transitions, and broader sample coverage.
• Complete AV1 hardening: film-grain policy, more tile/reference edge cases, transition stress, and long-session browser validation.
• Complete HEVC hardening: broader Main/Main10 samples, RPS/reference edge cases, resolution changes, and long-session browser validation. Keep HEVC Main12/P012, 4:2:2, 4:4:4, and SCC hidden until those paths are deliberately implemented.
• Revisit H.264 completeness: interlaced/field pictures, cropping, SPS/PPS edge cases, POC behavior, and conformance-stream coverage.

Deferred work:

• P012 export and 12-bit decode profiles.
• Non-4:2:0 profile families.
• VA-API encode support. H.264 encode is being developed separately; main keeps encode entrypoints hidden until packed headers, SPS/PPS/global-header handling, rate-control coverage, and application compatibility are ready.