feat(tools): vmaf-tune Phase A scaffold (ADR-0237) by lusoris · Pull Request #329 · lusoris/vmaf

lusoris · 2026-05-03T14:32:35Z

Summary

Scaffolds Phase A of tools/vmaf-tune/ per ADR-0237
(Phase A → Accepted; Phases B–F remain Proposed).
New Python tool drives FFmpeg over (preset, crf) × libx264, scores each
encode with the libvmaf CLI, emits a JSONL corpus row per cell. Codec
adapter registry is multi-codec from day one; only libx264 wired in
Phase A.
Schema is the API contract that Phase B (target-VMAF bisect) and
Phase C (per-title CRF predictor) will consume — exposed as
vmaftune.SCHEMA_VERSION (= 1) and vmaftune.CORPUS_ROW_KEYS.

Files

Surface	File
ADR	`docs/adr/0237-quality-aware-encode-automation.md` (status flipped)
Research digest	`docs/research/0044-quality-aware-encode-automation.md` (already in tree)
User docs	`docs/usage/vmaf-tune.md` (new)
Tool tree	`tools/vmaf-tune/` (new) — pyproject + src/vmaftune/{cli,encode,score,corpus,codec_adapters/x264}.py + tests + AGENTS.md
CHANGELOG	`changelog.d/added/T-VMAF-TUNE-phase-a-scaffold.md` + manual entry in `CHANGELOG.md`
Rebase note	`docs/rebase-notes.md` entry 0227
Pre-commit scope	`.pre-commit-config.yaml` extends black/isort/ruff to `tools/`
`.gitignore`	adds `corpus.jsonl` + tool build artefacts

Corpus JSONL row schema (v1)

schema_version | run_id | timestamp | src | src_sha256 |
width | height | pix_fmt | framerate | duration_s |
encoder | encoder_version | preset | crf | extra_params |
encode_path | encode_size_bytes | bitrate_kbps | encode_time_ms |
vmaf_score | vmaf_model | score_time_ms |
ffmpeg_version | vmaf_binary_version | exit_status

Six ADR-0108 deliverables

(1) Research digest: no new digest needed: — docs/research/0044-quality-aware-encode-automation.md
(shipped with the original Proposed ADR; cited as accepted here).
(2) Decision matrix: — ADR-0237 § Alternatives considered.
(3) AGENTS.md invariant note: — tools/vmaf-tune/AGENTS.md pins the JSONL
schema + codec-adapter contract as rebase-sensitive.
(4) Reproducer / smoke-test command: — pytest tools/vmaf-tune/tests/ -q (13 cases,
subprocess-mocked, no ffmpeg/vmaf binary required).
(5) CHANGELOG fragment: — fragment + manual Added entry (see commit 2 below).
(6) Rebase note: — docs/rebase-notes.md entry 0227.

Test plan

pytest tools/vmaf-tune/tests/ -q → 13 passed.
python tools/vmaf-tune/vmaf-tune --version → 0.0.1.
python tools/vmaf-tune/vmaf-tune corpus --help → renders argparse usage.
ruff check tools/vmaf-tune/ → clean.
pre-commit run --files <changed> → black/isort/ruff/yaml/secrets/semgrep all pass.
Real-binary corpus run deferred — Phase A is intentionally a scaffold;
smoke is mocked. A real corpus build takes hours and is the user's call
to schedule on the Netflix / KoNViD / BVI corpora.

Open questions

Whether to wire vmaf-tune as a meson-install target alongside
vmaf-perShot is left for a Phase A.1 follow-up; the standalone
Python install path (pip install -e tools/vmaf-tune) is enough for
the corpus-generation use case and avoids C-side meson churn while
the tool is still scaffolded.
The codec-adapter Protocol is intentionally lean (Phase A only
needs validate + name/range/default). Phase B+1 will widen it to
include build_command / parse_log / emit_per_shot_overrides
per the Research-0044 sketch.

What this PR does not do

No real corpus run.
No bisect / per-title / per-shot / ladder / MCP code (Phases B–F).
No upstream Netflix/vmaf path overlap; tools/vmaf-tune/ is wholly fork-local.
No libvmaf C-API change; no ffmpeg-patches change.

🤖 Generated with Claude Code

ADR-0237 Phase A is now Accepted; Phases B-F (target-VMAF bisect, per-title CRF predictor, per-shot dynamic CRF, Pareto ABR ladder, MCP tools) remain Proposed under the same umbrella ADR. Phase A ships a Python tool under tools/vmaf-tune/ that drives FFmpeg over a (preset, crf) grid against libx264, scores each encode with the libvmaf CLI, and emits a JSONL corpus of (source, encoder, params, bitrate, vmaf) rows. The corpus row schema is the API contract that Phase B / C will consume; it is versioned via vmaftune.SCHEMA_VERSION (currently 1) and exported as CORPUS_ROW_KEYS so downstream loaders can verify the shape programmatically. The codec-adapter registry under codec_adapters/ is multi-codec from day one per ADR-0237: Phase A wires libx264 only, but the harness (corpus.py, encode.py, score.py) routes through the registry without branching on codec identity, so subsequent codecs (libx265, libsvtav1, libvpx-vp9, libvvenc, neural extras) are one-file additions. Subprocess-mocked smoke tests under tools/vmaf-tune/tests/ cover command shape, version parsing, JSONL round-trip, encode-failure handling, and the schema contract; they require neither ffmpeg nor a built vmaf binary so CI runs them without backend dependencies. Six deep-dive deliverables (ADR-0108): 1. Research digest: docs/research/0044-quality-aware-encode-automation.md (already shipped with the Proposed ADR; cited as accepted here). 2. Decision matrix: ADR-0237 Alternatives considered table. 3. AGENTS.md invariant: tools/vmaf-tune/AGENTS.md pins the schema + adapter contracts as rebase-sensitive. 4. Reproducer: pytest tools/vmaf-tune/tests/ -q (13 cases pass with subprocess mocked). 5. CHANGELOG entry: changelog.d/added/T-VMAF-TUNE-phase-a-scaffold.md. 6. Rebase note: docs/rebase-notes.md entry 0227. User docs: docs/usage/vmaf-tune.md. No upstream Netflix/vmaf path overlap; tools/vmaf-tune/ is wholly fork-local. No libvmaf C-API or ffmpeg-patches surface change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ed content) Running scripts/release/concat-changelog-fragments.sh during the vmaf-tune Phase A scaffold commit re-rendered the Unreleased block from fragments only, which silently dropped 1262 lines of existing "Added/Changed" entries that had not yet been migrated to the fragment tree. Restore CHANGELOG.md to origin/master state and manually prepend the vmaf-tune Phase A entry to match the file's current edit pattern. The fragment under changelog.d/added/T-VMAF-TUNE-phase-a-scaffold.md is kept so a future re-run of the concat script after the broader fragment migration completes will keep the entry without drift. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Scaffolds Phase A of the new fork-local tools/vmaf-tune/ Python tool (per ADR-0237) to generate a (preset, crf) corpus by driving FFmpeg/libx264 and scoring encodes via the vmaf CLI, emitting a versioned JSONL schema intended for downstream Phase B/C consumers.

Changes:

Add tools/vmaf-tune/ package with CLI, codec-adapter registry (x264), encode/score drivers, corpus writer, and a mocked subprocess test suite.
Add user documentation and rebase note for the new tool; flip ADR-0237 status to “Accepted (Phase A only)”.
Extend pre-commit Python formatting/linting scope to tools/ and ignore vmaf-tune build/scratch artifacts.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tools/vmaf-tune/vmaf-tune	Repo-checkout console shim to run `vmaftune.cli:main` without install.
tools/vmaf-tune/tests/test_corpus.py	Mocked end-to-end smoke tests covering command shape, parsing, schema contract, and failure handling.
tools/vmaf-tune/src/vmaftune/score.py	`vmaf` CLI command builder + JSON parser + subprocess runner with timing/version capture.
tools/vmaf-tune/src/vmaftune/encode.py	FFmpeg command builder + subprocess runner with output sizing and version parsing helpers.
tools/vmaf-tune/src/vmaftune/corpus.py	Orchestrates grid sweeps, emits schema-validated rows, and writes JSONL.
tools/vmaf-tune/src/vmaftune/codec_adapters/x264.py	Phase A libx264 adapter (preset set + CRF range validation).
tools/vmaf-tune/src/vmaftune/codec_adapters/init.py	Adapter protocol + registry + lookup helpers (x264 only in Phase A).
tools/vmaf-tune/src/vmaftune/cli.py	Argparse wiring for `vmaf-tune corpus` subcommand.
tools/vmaf-tune/src/vmaftune/init.py	Public API constants: `__version__`, `SCHEMA_VERSION`, `CORPUS_ROW_KEYS`.
tools/vmaf-tune/pyproject.toml	Packaging metadata + pytest/ruff dev extras + console script entry point.
tools/vmaf-tune/README.md	Tool overview, layout, quickstart, and test instructions.
tools/vmaf-tune/AGENTS.md	Rebase-sensitive invariants (schema contract, adapter contract, subprocess seam).
docs/usage/vmaf-tune.md	User-facing docs for installation, CLI flags, and JSONL schema (Phase A).
docs/rebase-notes.md	Adds rebase note entry 0227 for the new tool and invariants.
docs/adr/README.md	Updates ADR index table content.
docs/adr/0237-quality-aware-encode-automation.md	Marks ADR-0237 as Accepted for Phase A (date/status metadata update).
changelog.d/added/T-VMAF-TUNE-phase-a-scaffold.md	Adds changelog fragment describing the new tool and Phase A scope.
CHANGELOG.md	Updates Unreleased content to include vmaf-tune entry (and other rendered content).
.pre-commit-config.yaml	Expands black/isort/ruff hooks to cover `tools/`.
.gitignore	Ignores vmaf-tune outputs and build artifacts (e.g. `corpus.jsonl`, dist/build dirs).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+
+Single-pass CRF-mode encodes; the eight standard x264 presets;
+quality range pinned to the canonical CRF window for which VMAF is
+informative. Out-of-range CRFs are accepted but logged.


+
+

+def _encode_path(opts: CorpusOptions, source: Path, preset: str, crf: int) -> Path:
+    stem = f"{source.stem}__{opts.encoder}__{preset}__crf{crf}.mp4"


+  under `tools/vmaf-tune/tests/` (13 cases) cover command shape,
+  version parsing, JSONL round-trip, encode-failure handling, and the
+  schema contract — no `ffmpeg` or built `vmaf` binary required.
+  User docs: [`docs/usage/vmaf-tune.md`](../docs/usage/vmaf-tune.md).


+| [ADR-0238](0238-vulkan-picture-preallocation.md) | T-VULKAN-PREALLOC — Vulkan VmafPicture preallocation surface that closes the API parity gap with CUDA / SYCL. Adds `VmafVulkanPicturePreallocationMethod` (NONE / HOST / DEVICE), `VmafVulkanPictureConfiguration`, `vmaf_vulkan_preallocate_pictures`, and `vmaf_vulkan_picture_fetch`. Mirrors the SYCL surface shape (chosen as the cleaner reference; CUDA's HOST_PINNED has no Vulkan analogue). DEVICE backs each picture's luma plane with a host-visible Vulkan buffer (VMA `AUTO_PREFER_HOST`); HOST uses `vmaf_picture_alloc`. Pool depth fixed at `pic_cnt = 2` (matches SYCL); growing it is additive. New `VMAF_PICTURE_BUFFER_TYPE_VULKAN_DEVICE` tag. New TU `libvmaf/src/vulkan/picture_vulkan_pool.c` (~180 LOC). Six smoke tests (`test_vulkan_pic_preallocation`) pin the contract under ASan / UBSan. Companion research digest: [`docs/research/0045-vulkan-picture-preallocation.md`](../research/0045-vulkan-picture-preallocation.md). | Proposed | vulkan, api, preallocation, fork-local, parity |
+| [ADR-0239](0239-gpu-picture-pool-dedup.md) | GPU dedup PR2 — promote `cuda/ring_buffer.{c,h}` out of `cuda/` to backend-agnostic `libvmaf/src/gpu_picture_pool.{c,h}`; rename `VmafRingBuffer*` → `VmafGpuPicturePool*` and `vmaf_ring_buffer_*` → `vmaf_gpu_picture_pool_*`. SYCL's `vmaf_sycl_picture_pool_*` keeps its public-internal API but delegates to the generic pool; `std::mutex` drops out. Vulkan migration deferred to a follow-up PR after #264 (ADR-0238) lands. The `Netflix#1300` mutex-destroy-order fix (ADR-0157) travels with the file unchanged. CPU build 47/47 pass, CUDA via CI (local nvcc has pre-existing include-path quirk). First "pool first" of the user's "all three sequenced — pool first, headers second, kernels third" GPU dedup roadmap. | Proposed | refactor, gpu, cuda, sycl, vulkan, dedup, fork-local |
+| [ADR-0240](0240-gpu-backend-pattern-doc.md) | GPU dedup PR3 — pattern doc instead of codegen. The original "headers second" scope (codegen `libvmaf_{cuda,sycl,vulkan,hip}.h` from one template) was revised after a 2026-05-02 audit found ~20 of ~200 lines per header truly shared (state lifecycle); the rest is backend-specific feature surface. Codegenning 10 % wasn't worth a Python build dependency. Ships [`docs/development/gpu-backend-template.md`](../development/gpu-backend-template.md) (recipe new GPU backends follow) + [`libvmaf/include/libvmaf/AGENTS.md`](../../libvmaf/include/libvmaf/AGENTS.md) (public-headers-tree invariant note pointing at the recipe). Locks the SYCL/Vulkan `NONE / HOST / DEVICE` three-method picture-preallocation convention as the new-backend default; CUDA's `HOST_PINNED` flagged as historical. Mirrors the tiny-AI ADR-0221 "recipe doc + shared helpers, not codegen" precedent. | Accepted | docs, gpu, agents, fork-local |
+| [ADR-0252](0252-ssimulacra2-host-xyb-simd.md) | ssimulacra2 Vulkan host-path AVX2 + NEON SIMD (T-GPU-OPT-VK-2). Adds `ssimulacra2_host_avx2.c` / `ssimulacra2_host_neon.c` with `plane_stride`-parameterised `linear_rgb_to_xyb` and `downsample_2x2` kernels; wires runtime dispatch into `ssimulacra2_vulkan.c`. Measured 2× XYB + 3.2× downsample speedup on 576×324 benchmark (cbrtf-bound, per-lane scalar stays for bit-exactness). AVX-512 omitted (<30% gain over AVX2 on cbrtf-dominated kernel). | Accepted | simd, vulkan, ssimulacra2, performance |


+- **`tools/vmaf-tune/` Phase A — quality-aware encode automation scaffold
+  (ADR-0237 Phase A Accepted, Research-0044).** New Python tool that
+  drives FFmpeg over a `(preset, crf)` grid against `libx264`, scores
+  each encode with the libvmaf CLI, and emits a JSONL corpus of
+  `(source, encoder, params, bitrate, vmaf)` rows. Schema versioned via
+  `vmaftune.SCHEMA_VERSION = 1` and exported as `CORPUS_ROW_KEYS`; the
+  schema is the API contract that Phase B (target-VMAF bisect) and
+  Phase C (per-title CRF predictor) will consume. Codec adapter
+  registry (`codec_adapters/`) is multi-codec from day one — Phase A
+  wires `libx264` only; subsequent codecs (`libx265`, `libsvtav1`,
+  `libvpx-vp9`, `libvvenc`, neural extras) are one-file additions
+  without touching the search loop. Subprocess-mocked smoke tests
+  under `tools/vmaf-tune/tests/` (13 cases) cover command shape,
+  version parsing, JSONL round-trip, encode-failure handling, and the
+  schema contract — no `ffmpeg` or built `vmaf` binary required.
+  User docs: [`docs/usage/vmaf-tune.md`](docs/usage/vmaf-tune.md).
+  Phases B–F remain Proposed under ADR-0237; this PR ships only the
+  Phase A corpus scaffold.
+


lusoris marked this pull request as ready for review May 3, 2026 17:38

Copilot AI review requested due to automatic review settings May 3, 2026 17:38

Lusoris and others added 2 commits May 3, 2026 19:38

lusoris force-pushed the feat/vmaf-tune-phase-a branch from 36cbd38 to 0785b1b Compare May 3, 2026 17:39

Copilot started reviewing on behalf of lusoris May 3, 2026 17:41 View session

Copilot AI reviewed May 3, 2026

View reviewed changes

Lusoris added 2 commits May 3, 2026 19:48

ci: re-trigger after canonical (N) checkbox prefix

483b8b5

ci: re-trigger after (1) untick (existing digest cited)

905dbcc

lusoris merged commit 826af8f into master May 3, 2026
54 checks passed

lusoris deleted the feat/vmaf-tune-phase-a branch May 3, 2026 18:15

This was referenced May 3, 2026

research(tools): vmaf-tune capability audit — what else can it do? #354

Merged

feat(tools): vmaf-tune — SVT-AV1 codec adapter #370

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(tools): vmaf-tune Phase A scaffold (ADR-0237)#329

feat(tools): vmaf-tune Phase A scaffold (ADR-0237)#329
lusoris merged 4 commits intomasterfrom
feat/vmaf-tune-phase-a

lusoris commented May 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		def _encode_path(opts: CorpusOptions, source: Path, preset: str, crf: int) -> Path:
		stem = f"{source.stem}__{opts.encoder}__{preset}__crf{crf}.mp4"

Uh oh!

Conversation

lusoris commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files

Corpus JSONL row schema (v1)

Six ADR-0108 deliverables

Test plan

Open questions

What this PR does not do

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lusoris commented May 3, 2026 •

edited

Loading