feat(tools): vmaf-tune — NVENC adapters (h264/hevc/av1)#364
Merged
Conversation
73fdc3e to
f3e7339
Compare
14 tasks
lusoris
pushed a commit
that referenced
this pull request
May 3, 2026
…s 17 adapters) Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)` and asks it for the FFmpeg argv slice via `adapter.ffmpeg_codec_args(preset, quality)` plus an optional `adapter.extra_params()`. Adapters that don't yet expose `ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so partial in-flight adapter PRs stay drivable end-to-end. `parse_versions(stderr, encoder=...)` selects a per-codec version probe (libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC, QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than raising. The `EncodeRequest.crf` field is preserved unchanged for the SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for adapter-side codec-agnostic vocabulary. Existing 13-test x264 suite still green; new 19-test multi-codec suite covers 9 representative codec shapes plus the unknown-codec / missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs (#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368 libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves) which can now drive end-to-end encodes without copying or mutating the harness. Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter contract" section, rebase-notes #228 invariant, CHANGELOG entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the NVIDIA NVENC family into tools/vmaf-tune/'s codec-adapter
registry: h264_nvenc, hevc_nvenc, av1_nvenc. Each codec ships its own
adapter file mirroring the X264Adapter shape; shared mnemonic-preset
mapping (ultrafast..placebo → p1..p7) and the [0, 51] constant-
quantizer window live in a private _nvenc_common.py helper.
Hardware encoders are 10-100× faster than software at the cost of
~3-5 VMAF points at matched bitrate; the rate-distortion curve is
codec-distinct, not codec-shifted, so they ship as separate codec
entries rather than a flag on libx264. known_codecs() now exposes
("av1_nvenc", "h264_nvenc", "hevc_nvenc", "libx264").
Test coverage in tools/vmaf-tune/tests/test_codec_adapter_nvenc.py:
preset name → p1..p7 mapping (parametrised), CQ range [0, 51]
validation, encoder-not-found subprocess simulation, and per-adapter
invariants. 58 new test cases; all 71 tool tests pass.
Documentation: docs/usage/vmaf-tune.md gains a "Hardware encoders
(NVENC)" section with the preset map and the software/hardware
trade-off note.
Six ADR-0108 deliverables:
1. Research digest: docs/research/0054-vmaf-tune-nvenc-adapters.md
2. Decision matrix: in ADR-0280 § Alternatives considered
3. AGENTS.md invariant: tools/vmaf-tune/AGENTS.md updated
4. Reproducer: cd tools/vmaf-tune && python -m pytest tests/ -q
5. CHANGELOG entry: under [Unreleased] § Added
6. Rebase note: docs/rebase-notes.md entry 0280
Follow-up filed in PR description and ADR-0280 § Consequences:
fr_regressor_v2's 6-slot codec one-hot needs expanding to ≥ 12
slots before training v2 on a corpus that includes the hardware
codecs (separate PR; gated on training corpus availability).
Refs: ADR-0280, ADR-0237 (parent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
f3e7339 to
d1a8a80
Compare
There was a problem hiding this comment.
Pull request overview
Adds NVIDIA NVENC codec adapters to the tools/vmaf-tune/ Phase-A codec-adapter registry so the corpus generator can sweep hardware encoders (h264_nvenc, hevc_nvenc, av1_nvenc) using the same adapter contract as software encoders.
Changes:
- Introduces three new NVENC codec adapters plus a shared
_nvenc_common.pyhelper for preset + CQ validation. - Expands the codec-adapter registry and adds a dedicated parametrized pytest suite for NVENC adapter behavior.
- Updates fork documentation/records (ADR + research digest + rebase notes + usage docs + changelog) to describe the new adapters.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/vmaf-tune/src/vmaftune/codec_adapters/_nvenc_common.py | Shared NVENC preset mapping and CQ validation helpers. |
| tools/vmaf-tune/src/vmaftune/codec_adapters/{h264_nvenc,hevc_nvenc,av1_nvenc}.py | New per-codec NVENC adapter dataclasses wired to the shared helper. |
| tools/vmaf-tune/src/vmaftune/codec_adapters/init.py | Registers NVENC adapters in the Phase-A codec registry. |
| tools/vmaf-tune/tests/test_codec_adapter_nvenc.py | New parametrized smoke/contract tests for NVENC adapters. |
| tools/vmaf-tune/tests/test_corpus.py | Updates Phase-A registry assertions to include NVENC codecs. |
| tools/vmaf-tune/AGENTS.md | Notes invariants/expectations for codec adapters and shared helpers. |
| docs/usage/vmaf-tune.md | Adds an NVENC usage section (presets, CQ window, trade-offs). |
| docs/research/0065-vmaf-tune-nvenc-adapters.md | New research digest describing design rationale. |
| docs/adr/0290-vmaf-tune-nvenc-adapters.md | New ADR capturing the decision and consequences. |
| docs/adr/README.md | Adds ADR index row for ADR-0290. |
| docs/rebase-notes.md | Adds a fork-local rebase note entry for this change. |
| CHANGELOG.md | Adds an Unreleased entry describing the NVENC adapters. |
Comments suppressed due to low confidence (1)
tools/vmaf-tune/src/vmaftune/codec_adapters/init.py:44
CodecAdapteris used as the return type ofget_adapter(), but theProtocolcurrently only declares data attributes; it doesn't declare thevalidate()method thatcorpus.iter_rows()calls on every adapter. This makes type checking / IDE usage inconsistent. Please adddef validate(self, preset: str, value: int) -> None:(and optionallypresets) to the protocol so it matches the adapter contract actually relied on by the harness.
>>>>>>> f3e73397 (feat(tools): vmaf-tune — NVENC adapters (h264/hevc/av1))
from .x264 import X264Adapter
from .x265 import X265Adapter
class CodecAdapter(Protocol):
"""Phase A codec-adapter contract (subset of the ADR-0237 sketch).
Phase B+ extends with two-pass / log-parsing / per-shot emit hooks.
"""
name: str
encoder: str
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -264,6 +264,7 @@ ADRs may exist there for local session continuity, but the tracked | |||
| | [ADR-0259](0259-hip-third-consumer-ciede.md) | T7-10b third-consumer PR — `ciede_hip` host scaffolding via the kernel-template mirror established by [ADR-0241](0241-hip-first-consumer-psnr.md). Ships [`libvmaf/src/feature/hip/ciede_hip.{c,h}`](../../libvmaf/src/feature/hip/ciede_hip.c) — mirrors `libvmaf/src/feature/cuda/integer_ciede_cuda.c`'s init/submit/collect/close call graph verbatim, including the **intentional bypass** of `submit_pre_launch` (ciede's kernel writes one float per block, no atomic, no memset required). Same scaffold posture as ADR-0241 / ADR-0254: registration succeeds, `init()` returns `-ENOSYS` until T7-10b flips the kernel-template helper bodies to real HIP calls. New `vmaf_fex_ciede_hip` row in `feature_extractor_list` under `#if HAVE_HIP`; `VMAF_FEATURE_EXTRACTOR_HIP` flag stays cleared. Smoke test grows by one sub-test (`test_ciede_hip_extractor_registered`). Pins the kernel-template's "no-memset bypass" path so the runtime PR can flip helper bodies without inventing a new template variant for ciede. Picks `integer_ciede_cuda` (243 LOC) over `integer_motion_cuda` (503 LOC, stateful) and `float_ansnr_cuda` (298 LOC, duplicates ADR-0254's precision posture). | Accepted | gpu, hip, rocm, amd, kernel-template, fork-local | | |||
| | [ADR-0272](0272-fr-regressor-v2-codec-aware-scaffold.md) | `fr_regressor_v2` codec-aware scaffold — first downstream consumer of the vmaf-tune Phase A JSONL corpus ([ADR-0237](0237-quality-aware-encode-automation.md)). Ships [`ai/scripts/train_fr_regressor_v2.py`](../../ai/scripts/train_fr_regressor_v2.py), a smoke ONNX (`fr_regressor_v2.onnx` registered with `smoke: true`), sidecar JSON, and full doc surface ([model card](../ai/models/fr_regressor_v2.md), [research digest](../research/0058-fr-regressor-v2-feasibility.md)). Two-input ONNX: 6 canonical libvmaf features (`adm2`, `vif_scale0..3`, `motion2`, StandardScaler-normalised) + 8-D codec block (6-way encoder one-hot + preset_norm + crf_norm, both in `[0, 1]`). MLP shape `6 -> 16 -> 16 -> 1` with codec block concatenated before the first dense layer (matches the existing `FRRegressor(num_codecs=8)` plumbing landed by [ADR-0235](0235-codec-aware-fr-regressor.md)). Registry row stays `smoke: true` until a follow-up PR (T7-FR-REGRESSOR-V2-PROD) re-runs training on a real Phase A corpus and clears v1's 0.95 LOSO PLCC ship gate with the ≥0.005 multi-codec lift required by ADR-0235. | Proposed | ai, dnn, tiny-ai, fr-regressor, codec-aware, vmaf-tune, fork-local | | |||
Comment on lines
+184
to
+204
| - **`vmaf-tune` NVENC codec adapters — `h264_nvenc`, `hevc_nvenc`, | ||
| `av1_nvenc` (ADR-0290).** Wires the NVIDIA NVENC family into | ||
| `tools/vmaf-tune/src/vmaftune/codec_adapters/` as three new | ||
| adapter files plus a shared `_nvenc_common.py` helper that owns | ||
| the mnemonic preset map (`ultrafast`..`placebo` → `p1`..`p7`) | ||
| and the constant-quantizer hard window `[0, 51]`. Hardware | ||
| encoders are 10–100× faster than software at the cost of ~3–5 | ||
| VMAF points at matched bitrate; the R-D curve is codec-distinct | ||
| not codec-shifted, so they ship as separate codec entries rather | ||
| than a flag on `libx264`. Registry now exposes | ||
| `known_codecs() == ("av1_nvenc", "h264_nvenc", "hevc_nvenc", | ||
| "libx264")`. Test coverage: parametrised suite under | ||
| `tools/vmaf-tune/tests/test_codec_adapter_nvenc.py` (preset | ||
| mapping, CQ range validation, encoder-not-found subprocess | ||
| simulation). Documentation: | ||
| [`docs/usage/vmaf-tune.md`](docs/usage/vmaf-tune.md) "Hardware | ||
| encoders (NVENC)" section. Follow-up filed: `fr_regressor_v2` | ||
| codec one-hot expansion from 6 → ≥ 12 slots before training v2 | ||
| on a corpus that includes hardware codecs. Companion research | ||
| digest: | ||
| [`docs/research/0065-vmaf-tune-nvenc-adapters.md`](docs/research/0065-vmaf-tune-nvenc-adapters.md). |
Comment on lines
+1
to
+6
| # Research-0054: NVENC codec adapters for `vmaf-tune` — option-space digest | ||
|
|
||
| - **Date**: 2026-05-03 | ||
| - **Companion ADR**: [ADR-0290](../adr/0290-vmaf-tune-nvenc-adapters.md) | ||
| - **Status**: Snapshot at proposal time; the implementation PR | ||
| supersedes the operational details. |
lusoris
pushed a commit
that referenced
this pull request
May 5, 2026
…s 17 adapters) Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)` and asks it for the FFmpeg argv slice via `adapter.ffmpeg_codec_args(preset, quality)` plus an optional `adapter.extra_params()`. Adapters that don't yet expose `ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so partial in-flight adapter PRs stay drivable end-to-end. `parse_versions(stderr, encoder=...)` selects a per-codec version probe (libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC, QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than raising. The `EncodeRequest.crf` field is preserved unchanged for the SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for adapter-side codec-agnostic vocabulary. Existing 13-test x264 suite still green; new 19-test multi-codec suite covers 9 representative codec shapes plus the unknown-codec / missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs (#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368 libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves) which can now drive end-to-end encodes without copying or mutating the harness. Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter contract" section, rebase-notes #228 invariant, CHANGELOG entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
May 5, 2026
…s 17 adapters) (#376) * feat(tools): vmaf-tune encode.py — codec-agnostic dispatcher (unblocks 17 adapters) Refactors `tools/vmaf-tune/src/vmaftune/encode.py` away from the Phase A hard-coded `libx264` `-c:v / -preset / -crf` argv. `run_encode` now looks up the codec adapter via `codec_adapters.get_adapter(req.encoder)` and asks it for the FFmpeg argv slice via `adapter.ffmpeg_codec_args(preset, quality)` plus an optional `adapter.extra_params()`. Adapters that don't yet expose `ffmpeg_codec_args` fall back silently to the legacy x264-CRF shape so partial in-flight adapter PRs stay drivable end-to-end. `parse_versions(stderr, encoder=...)` selects a per-codec version probe (libx264, libx265, libsvtav1, libvpx-vp9, libaom-av1, libvvenc, NVENC, QSV, AMF, VideoToolbox); unknown encoders return "unknown" rather than raising. The `EncodeRequest.crf` field is preserved unchanged for the SCHEMA_VERSION=1 row contract; a `quality` property mirrors it for adapter-side codec-agnostic vocabulary. Existing 13-test x264 suite still green; new 19-test multi-codec suite covers 9 representative codec shapes plus the unknown-codec / missing-method fallback paths. Unblocks 17 in-flight codec adapter PRs (#360 libaom, #362 libx265, #364 NVENC, #366 AMF, #367 QSV, #368 libvvenc, #370 libsvtav1, #373 VideoToolbox, plus follow-on waves) which can now drive end-to-end encodes without copying or mutating the harness. Ships ADR-0294 + research digest 0054, vmaf-tune.md "Codec adapter contract" section, rebase-notes #228 invariant, CHANGELOG entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(docs): renumber encode-multi-codec ADR 0294→0297 + research 0069→0070 --------- Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
h264_nvenc,hevc_nvenc,av1_nvenc) totools/vmaf-tune/'s codec-adapter registry. Hardwareencoders are 10–100× faster than software at the cost of ~3–5 VMAF
points at matched bitrate; the rate-distortion curve is
codec-distinct, not codec-shifted, which is why NVENC ships as
separate codec entries rather than a flag on the existing software
adapters.
ultrafast→p1,medium→p4,placebo→p7, …) and the[0, 51]constant-quantizer windowlive in
tools/vmaf-tune/src/vmaftune/codec_adapters/_nvenc_common.pyso the per-codec adapter files stay thin.
docs/usage/vmaf-tune.mdgains a "Hardware encoders(NVENC)" section with the preset map and the speed/quality
trade-off note.
Codec one-hot follow-up (separate PR)
fr_regressor_v2currently uses a 6-slot codec one-hot. Addingthe three NVENC codecs (and the parallel software-codec adapter PRs
for x265 / svtav1 / libaom) pushes the natural slot count to ≥ 9,
with headroom for ≥ 12 once QSV / VAAPI follow.
The schema needs extending before training v2 on a corpus that
includes hardware codecs. Tracking this as a follow-up:
new codecs and is out of scope for this PR.
Six ADR-0108 deliverables
docs/research/0065-vmaf-tune-nvenc-adapters.md.docs/adr/0290-vmaf-tune-nvenc-adapters.md§ Alternatives considered.tools/vmaf-tune/AGENTS.mdupdated to call out the NVENC family + the_nvenc_common.pyshared-helper convention.cd tools/vmaf-tune && python -m pytest tests/ -q.[Unreleased] § AddedinCHANGELOG.md.0290indocs/rebase-notes.md. Wholly fork-local — no upstream Netflix/vmaf path overlap.Test plan
cd tools/vmaf-tune && python -m pytest tests/ -q(71 tests pass).cd tools/vmaf-tune && python -m pytest tests/test_codec_adapter_nvenc.py -v(58-case parametrised suite).from vmaftune.codec_adapters import known_codecs, get_adapter; assert "h264_nvenc" in known_codecs(); assert get_adapter("h264_nvenc").nvenc_preset("medium") == "p4".ruff check tools/vmaf-tune/clean; pre-commit black/isort/ruff clean.docs/usage/vmaf-tune.md"Hardware encoders (NVENC)" section visible.Hardware availability
h264_nvenc: NVIDIA Kepler+ (most modern GPUs).hevc_nvenc: NVIDIA Maxwell 2nd-gen+ (GTX 960+).av1_nvenc: NVIDIA Ada Lovelace+ (RTX 40-series, L40, L4).When FFmpeg returns
Encoder X not found, the harness recordsexit_status != 0and skips scoring — partial corpora over aheterogeneous fleet stay well-formed.
Refs
vmaf-tuneumbrella spec)docs/research/0065-vmaf-tune-nvenc-adapters.md🤖 Generated with Claude Code