Skip to content

feat(tools): vmaf-tune — NVENC adapters (h264/hevc/av1)#364

Merged
lusoris merged 3 commits intomasterfrom
feat/vmaf-tune-codec-adapter-nvenc
May 5, 2026
Merged

feat(tools): vmaf-tune — NVENC adapters (h264/hevc/av1)#364
lusoris merged 3 commits intomasterfrom
feat/vmaf-tune-codec-adapter-nvenc

Conversation

@lusoris
Copy link
Copy Markdown
Owner

@lusoris lusoris commented May 3, 2026

Summary

  • Adds three NVIDIA NVENC codec adapters (h264_nvenc, hevc_nvenc,
    av1_nvenc) to tools/vmaf-tune/'s codec-adapter registry. Hardware
    encoders are 10–100× faster than software at the cost of ~3–5 VMAF
    points at matched bitrate; the rate-distortion curve is
    codec-distinct, not codec-shifted, which is why NVENC ships as
    separate codec entries rather than a flag on the existing software
    adapters.
  • Shared mnemonic preset map (ultrafastp1, mediump4,
    placebop7, …) and the [0, 51] constant-quantizer window
    live in tools/vmaf-tune/src/vmaftune/codec_adapters/_nvenc_common.py
    so the per-codec adapter files stay thin.
  • Documentation: docs/usage/vmaf-tune.md gains a "Hardware encoders
    (NVENC)" section with the preset map and the speed/quality
    trade-off note.

Codec one-hot follow-up (separate PR)

fr_regressor_v2 currently uses a 6-slot codec one-hot. Adding
the three NVENC codecs (and the parallel software-codec adapter PRs
for x265 / svtav1 / libaom) pushes the natural slot count to ≥ 9,
with headroom for ≥ 12 once QSV / VAAPI follow.

The schema needs extending before training v2 on a corpus that
includes hardware codecs
. Tracking this as a follow-up:

  • The codec-adapter contract is unblocked here (this PR).
  • The schema bump is gated on training corpus availability for the
    new codecs and is out of scope for this PR.
  • ADR-0280 § Consequences captures the dependency.

Six ADR-0108 deliverables

Test plan

  • cd tools/vmaf-tune && python -m pytest tests/ -q (71 tests pass).
  • cd tools/vmaf-tune && python -m pytest tests/test_codec_adapter_nvenc.py -v (58-case parametrised suite).
  • Smoke check: from vmaftune.codec_adapters import known_codecs, get_adapter; assert "h264_nvenc" in known_codecs(); assert get_adapter("h264_nvenc").nvenc_preset("medium") == "p4".
  • Linters: ruff check tools/vmaf-tune/ clean; pre-commit black/isort/ruff clean.
  • Docs render: docs/usage/vmaf-tune.md "Hardware encoders (NVENC)" section visible.

Hardware availability

  • h264_nvenc: NVIDIA Kepler+ (most modern GPUs).
  • hevc_nvenc: NVIDIA Maxwell 2nd-gen+ (GTX 960+).
  • av1_nvenc: NVIDIA Ada Lovelace+ (RTX 40-series, L40, L4).

When FFmpeg returns Encoder X not found, the harness records
exit_status != 0 and skips scoring — partial corpora over a
heterogeneous fleet stay well-formed.

Refs

🤖 Generated with Claude Code

@lusoris lusoris force-pushed the feat/vmaf-tune-codec-adapter-nvenc branch from 73fdc3e to f3e7339 Compare May 3, 2026 19:38
lusoris pushed a commit that referenced this pull request May 3, 2026
…s 17 adapters)

Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A
hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now
looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)`
and asks it for the FFmpeg argv slice via
`adapter.ffmpeg_codec_args(preset, quality)` plus an optional
`adapter.extra_params()`. Adapters that don't yet expose
`ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so
partial in-flight adapter PRs stay drivable end-to-end.
`parse_versions(stderr, encoder=...)` selects a per-codec version probe
(libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC,
QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than
raising. The `EncodeRequest.crf` field is preserved unchanged for the
SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for
adapter-side codec-agnostic vocabulary.

Existing 13-test x264 suite still green; new 19-test multi-codec suite
covers 9 representative codec shapes plus the unknown-codec /
missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs
(#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368
libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves)
which can now drive end-to-end encodes without copying or mutating the
harness.

Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter
contract" section, rebase-notes #228 invariant, CHANGELOG entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lusoris lusoris marked this pull request as ready for review May 5, 2026 08:34
Copilot AI review requested due to automatic review settings May 5, 2026 08:34
Wires the NVIDIA NVENC family into tools/vmaf-tune/'s codec-adapter
registry: h264_nvenc, hevc_nvenc, av1_nvenc. Each codec ships its own
adapter file mirroring the X264Adapter shape; shared mnemonic-preset
mapping (ultrafast..placebo → p1..p7) and the [0, 51] constant-
quantizer window live in a private _nvenc_common.py helper.

Hardware encoders are 10-100× faster than software at the cost of
~3-5 VMAF points at matched bitrate; the rate-distortion curve is
codec-distinct, not codec-shifted, so they ship as separate codec
entries rather than a flag on libx264. known_codecs() now exposes
("av1_nvenc", "h264_nvenc", "hevc_nvenc", "libx264").

Test coverage in tools/vmaf-tune/tests/test_codec_adapter_nvenc.py:
preset name → p1..p7 mapping (parametrised), CQ range [0, 51]
validation, encoder-not-found subprocess simulation, and per-adapter
invariants. 58 new test cases; all 71 tool tests pass.

Documentation: docs/usage/vmaf-tune.md gains a "Hardware encoders
(NVENC)" section with the preset map and the software/hardware
trade-off note.

Six ADR-0108 deliverables:
  1. Research digest: docs/research/0054-vmaf-tune-nvenc-adapters.md
  2. Decision matrix: in ADR-0280 § Alternatives considered
  3. AGENTS.md invariant: tools/vmaf-tune/AGENTS.md updated
  4. Reproducer: cd tools/vmaf-tune && python -m pytest tests/ -q
  5. CHANGELOG entry: under [Unreleased] § Added
  6. Rebase note: docs/rebase-notes.md entry 0280

Follow-up filed in PR description and ADR-0280 § Consequences:
fr_regressor_v2's 6-slot codec one-hot needs expanding to ≥ 12
slots before training v2 on a corpus that includes the hardware
codecs (separate PR; gated on training corpus availability).

Refs: ADR-0280, ADR-0237 (parent).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lusoris lusoris force-pushed the feat/vmaf-tune-codec-adapter-nvenc branch from f3e7339 to d1a8a80 Compare May 5, 2026 08:35
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds NVIDIA NVENC codec adapters to the tools/vmaf-tune/ Phase-A codec-adapter registry so the corpus generator can sweep hardware encoders (h264_nvenc, hevc_nvenc, av1_nvenc) using the same adapter contract as software encoders.

Changes:

  • Introduces three new NVENC codec adapters plus a shared _nvenc_common.py helper for preset + CQ validation.
  • Expands the codec-adapter registry and adds a dedicated parametrized pytest suite for NVENC adapter behavior.
  • Updates fork documentation/records (ADR + research digest + rebase notes + usage docs + changelog) to describe the new adapters.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/vmaf-tune/src/vmaftune/codec_adapters/_nvenc_common.py Shared NVENC preset mapping and CQ validation helpers.
tools/vmaf-tune/src/vmaftune/codec_adapters/{h264_nvenc,hevc_nvenc,av1_nvenc}.py New per-codec NVENC adapter dataclasses wired to the shared helper.
tools/vmaf-tune/src/vmaftune/codec_adapters/init.py Registers NVENC adapters in the Phase-A codec registry.
tools/vmaf-tune/tests/test_codec_adapter_nvenc.py New parametrized smoke/contract tests for NVENC adapters.
tools/vmaf-tune/tests/test_corpus.py Updates Phase-A registry assertions to include NVENC codecs.
tools/vmaf-tune/AGENTS.md Notes invariants/expectations for codec adapters and shared helpers.
docs/usage/vmaf-tune.md Adds an NVENC usage section (presets, CQ window, trade-offs).
docs/research/0065-vmaf-tune-nvenc-adapters.md New research digest describing design rationale.
docs/adr/0290-vmaf-tune-nvenc-adapters.md New ADR capturing the decision and consequences.
docs/adr/README.md Adds ADR index row for ADR-0290.
docs/rebase-notes.md Adds a fork-local rebase note entry for this change.
CHANGELOG.md Adds an Unreleased entry describing the NVENC adapters.
Comments suppressed due to low confidence (1)

tools/vmaf-tune/src/vmaftune/codec_adapters/init.py:44

  • CodecAdapter is used as the return type of get_adapter(), but the Protocol currently only declares data attributes; it doesn't declare the validate() method that corpus.iter_rows() calls on every adapter. This makes type checking / IDE usage inconsistent. Please add def validate(self, preset: str, value: int) -> None: (and optionally presets) to the protocol so it matches the adapter contract actually relied on by the harness.
>>>>>>> f3e73397 (feat(tools): vmaf-tune — NVENC adapters (h264/hevc/av1))
from .x264 import X264Adapter
from .x265 import X265Adapter


class CodecAdapter(Protocol):
    """Phase A codec-adapter contract (subset of the ADR-0237 sketch).

    Phase B+ extends with two-pass / log-parsing / per-shot emit hooks.
    """

    name: str
    encoder: str

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/adr/README.md
@@ -264,6 +264,7 @@ ADRs may exist there for local session continuity, but the tracked
| [ADR-0259](0259-hip-third-consumer-ciede.md) | T7-10b third-consumer PR — `ciede_hip` host scaffolding via the kernel-template mirror established by [ADR-0241](0241-hip-first-consumer-psnr.md). Ships [`libvmaf/src/feature/hip/ciede_hip.{c,h}`](../../libvmaf/src/feature/hip/ciede_hip.c) — mirrors `libvmaf/src/feature/cuda/integer_ciede_cuda.c`'s init/submit/collect/close call graph verbatim, including the **intentional bypass** of `submit_pre_launch` (ciede's kernel writes one float per block, no atomic, no memset required). Same scaffold posture as ADR-0241 / ADR-0254: registration succeeds, `init()` returns `-ENOSYS` until T7-10b flips the kernel-template helper bodies to real HIP calls. New `vmaf_fex_ciede_hip` row in `feature_extractor_list` under `#if HAVE_HIP`; `VMAF_FEATURE_EXTRACTOR_HIP` flag stays cleared. Smoke test grows by one sub-test (`test_ciede_hip_extractor_registered`). Pins the kernel-template's "no-memset bypass" path so the runtime PR can flip helper bodies without inventing a new template variant for ciede. Picks `integer_ciede_cuda` (243 LOC) over `integer_motion_cuda` (503 LOC, stateful) and `float_ansnr_cuda` (298 LOC, duplicates ADR-0254's precision posture). | Accepted | gpu, hip, rocm, amd, kernel-template, fork-local |
| [ADR-0272](0272-fr-regressor-v2-codec-aware-scaffold.md) | `fr_regressor_v2` codec-aware scaffold — first downstream consumer of the vmaf-tune Phase A JSONL corpus ([ADR-0237](0237-quality-aware-encode-automation.md)). Ships [`ai/scripts/train_fr_regressor_v2.py`](../../ai/scripts/train_fr_regressor_v2.py), a smoke ONNX (`fr_regressor_v2.onnx` registered with `smoke: true`), sidecar JSON, and full doc surface ([model card](../ai/models/fr_regressor_v2.md), [research digest](../research/0058-fr-regressor-v2-feasibility.md)). Two-input ONNX: 6 canonical libvmaf features (`adm2`, `vif_scale0..3`, `motion2`, StandardScaler-normalised) + 8-D codec block (6-way encoder one-hot + preset_norm + crf_norm, both in `[0, 1]`). MLP shape `6 -> 16 -> 16 -> 1` with codec block concatenated before the first dense layer (matches the existing `FRRegressor(num_codecs=8)` plumbing landed by [ADR-0235](0235-codec-aware-fr-regressor.md)). Registry row stays `smoke: true` until a follow-up PR (T7-FR-REGRESSOR-V2-PROD) re-runs training on a real Phase A corpus and clears v1's 0.95 LOSO PLCC ship gate with the ≥0.005 multi-codec lift required by ADR-0235. | Proposed | ai, dnn, tiny-ai, fr-regressor, codec-aware, vmaf-tune, fork-local |
Comment thread CHANGELOG.md
Comment on lines +184 to +204
- **`vmaf-tune` NVENC codec adapters — `h264_nvenc`, `hevc_nvenc`,
`av1_nvenc` (ADR-0290).** Wires the NVIDIA NVENC family into
`tools/vmaf-tune/src/vmaftune/codec_adapters/` as three new
adapter files plus a shared `_nvenc_common.py` helper that owns
the mnemonic preset map (`ultrafast`..`placebo` → `p1`..`p7`)
and the constant-quantizer hard window `[0, 51]`. Hardware
encoders are 10–100× faster than software at the cost of ~3–5
VMAF points at matched bitrate; the R-D curve is codec-distinct
not codec-shifted, so they ship as separate codec entries rather
than a flag on `libx264`. Registry now exposes
`known_codecs() == ("av1_nvenc", "h264_nvenc", "hevc_nvenc",
"libx264")`. Test coverage: parametrised suite under
`tools/vmaf-tune/tests/test_codec_adapter_nvenc.py` (preset
mapping, CQ range validation, encoder-not-found subprocess
simulation). Documentation:
[`docs/usage/vmaf-tune.md`](docs/usage/vmaf-tune.md) "Hardware
encoders (NVENC)" section. Follow-up filed: `fr_regressor_v2`
codec one-hot expansion from 6 → ≥ 12 slots before training v2
on a corpus that includes hardware codecs. Companion research
digest:
[`docs/research/0065-vmaf-tune-nvenc-adapters.md`](docs/research/0065-vmaf-tune-nvenc-adapters.md).
Comment on lines +1 to +6
# Research-0054: NVENC codec adapters for `vmaf-tune` — option-space digest

- **Date**: 2026-05-03
- **Companion ADR**: [ADR-0290](../adr/0290-vmaf-tune-nvenc-adapters.md)
- **Status**: Snapshot at proposal time; the implementation PR
supersedes the operational details.
@lusoris lusoris merged commit 6e436f5 into master May 5, 2026
54 checks passed
@lusoris lusoris deleted the feat/vmaf-tune-codec-adapter-nvenc branch May 5, 2026 09:08
lusoris pushed a commit that referenced this pull request May 5, 2026
…s 17 adapters)

Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A
hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now
looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)`
and asks it for the FFmpeg argv slice via
`adapter.ffmpeg_codec_args(preset, quality)` plus an optional
`adapter.extra_params()`. Adapters that don't yet expose
`ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so
partial in-flight adapter PRs stay drivable end-to-end.
`parse_versions(stderr, encoder=...)` selects a per-codec version probe
(libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC,
QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than
raising. The `EncodeRequest.crf` field is preserved unchanged for the
SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for
adapter-side codec-agnostic vocabulary.

Existing 13-test x264 suite still green; new 19-test multi-codec suite
covers 9 representative codec shapes plus the unknown-codec /
missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs
(#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368
libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves)
which can now drive end-to-end encodes without copying or mutating the
harness.

Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter
contract" section, rebase-notes #228 invariant, CHANGELOG entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lusoris added a commit that referenced this pull request May 5, 2026
…s 17 adapters) (#376)

* feat(tools): vmaf-tune encode.py — codec-agnostic dispatcher (unblocks 17 adapters)

Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A
hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now
looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)`
and asks it for the FFmpeg argv slice via
`adapter.ffmpeg_codec_args(preset, quality)` plus an optional
`adapter.extra_params()`. Adapters that don't yet expose
`ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so
partial in-flight adapter PRs stay drivable end-to-end.
`parse_versions(stderr, encoder=...)` selects a per-codec version probe
(libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC,
QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than
raising. The `EncodeRequest.crf` field is preserved unchanged for the
SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for
adapter-side codec-agnostic vocabulary.

Existing 13-test x264 suite still green; new 19-test multi-codec suite
covers 9 representative codec shapes plus the unknown-codec /
missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs
(#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368
libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves)
which can now drive end-to-end encodes without copying or mutating the
harness.

Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter
contract" section, rebase-notes #228 invariant, CHANGELOG entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(docs): renumber encode-multi-codec ADR 0294→0297 + research 0069→0070

---------

Co-authored-by: Lusoris <lusoris@pm.me>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants