Skip to content

feat: add mp4ff-mvhevc command line tool#515

Merged
tobbee merged 5 commits into
masterfrom
feat-mvhevc-cli
Jul 7, 2026
Merged

feat: add mp4ff-mvhevc command line tool#515
tobbee merged 5 commits into
masterfrom
feat-mvhevc-cli

Conversation

@tobbee

@tobbee tobbee commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

The capstone of the MV-HEVC effort: a command line tool that ties the VPS parser, lhvC, oinf/linf builders, sample entries and spatial-video boxes together.

mp4ff-mvhevc has two subcommands, structured like the other mp4ff-* tools (mainrun(os.Args, os.Stdout), an options struct + parseOptions per subcommand, internal.GetVersion()):

  • info <input.mp4> — displays the MV-HEVC structure: sample entries, hvcC/lhvC, Dolby Vision configuration (dvcC/dvvC/dvwC), oinf/linf sample groups, vexu/hfov spatial metadata, and trgr (including unregistered track-group types).
  • add [options] <input.hevc|mp4> <output.mp4> — muxes HEVC into a single-track MV-HEVC mp4:
    • Annex B (with -fps) or an existing HEVC/MV-HEVC mp4 as input.
    • hvc1 with hvcC (and lhvC when an enhancement layer is present).
    • oinf/linf sgpd+sbgp for layered (multi-view) input.
    • optional Apple spatial-video metadata (vexu/hfov) via -spatial (with -baseline, -hfov, -hero, -reversed).

Registered in the README, .goreleaser.yml and the Makefile.

Correctness (adapted from the reference implementation)

A focused review of the muxer surfaced several issues that are fixed here:

  • Access-unit segmentation splits at the first slice of each base-layer picture (first_slice_segment_in_pic_flag), so a multi-slice picture is no longer split across multiple samples.
  • ctts (composition offsets / B-frame reordering) is carried over when re-muxing an mp4 input — verified byte-identical to the source on mp4/testdata/ed_hevc.mp4.
  • A conformant vmhd (flags = 1) is written via mp4.CreateVmhd().
  • Only base-layer prefix SEI is hoisted into hvcC (suffix SEI is no longer mislabeled as prefix).
  • The single chunk offset is range-checked; output larger than 32-bit stco offsets fails cleanly (co64 is not emitted).
  • The cstg track group the reference writes is not written. cstg is the complete subset track grouping of ISO/IEC 14496-15 §9.5.1 (GPAC writes it with track_group_id = 1000 + track_id when muxing layered HEVC, which is where the reference's hardcoded 1001 comes from), but it marks the minimal set of multiple tracks that together carry the complete L-HEVC bitstream — for the single-track output of this tool it conveys nothing, and Apple's own muxer writes no track group either. The cstg type is now registered on TrackGroupTypeBox (alongside msrc/ster), so info decodes and displays it in files that have it.

Tests

Unit tests for the helpers, plus end-to-end addinfo round-trips:

  • Single-layer paths on the repo's existing HEVC assets: an Annex B stream with -spatial (asserts the vexu/hfov/stri output) and an mp4 re-mux (asserts hvc1 and that ctts is carried over).
  • Multilayer Annex B path: a synthetic MV-HEVC stream built in-test (real stereo VPS with vps_extension, base + enhancement parameter sets, placeholder slice payloads — mp4ff only touches boxes) is muxed and asserted to produce lhvC + oinf/linf.
  • New testdata cmd/mp4ff-mvhevc/testdata/stereo_spatial.mp4 (4.9 KB): a genuine MV-HEVC spatial video encoded with Apple VideoToolbox from synthetic gradient frames by the checked-in generator testdata/gen_stereo_spatial.swift (entirely own content, so redistributable; ffmpeg confirms view_ids_available=0,1). Since it is muxed by an independent implementation, it tests info against externally-produced boxes (hvcC+lhvC, vexu with stri/hero/blin, hfov, ctts, and no oinf/linf), the -idr flag, and drives the multilayer mp4-input re-mux path end-to-end: parameter sets picked up from hvcC+lhvC, oinf/linf built from a real VideoToolbox VPS, and ctts carry-over.
  • Dolby Vision and track-group display: a dvwC box and a trgr with registered (ster, cstg) and unregistered group types are attached in-test and asserted in the info output; cstg round-trip is covered in mp4/trgr_test.go.
  • Error/version paths covered. go test ./..., go vet, golangci-lint clean.

The tool has also been validated locally against MV-HEVC files that cannot be checked in (Apple/GPAC-produced spatial videos, Dolby Vision profile 20, and a stream with enhancement-layer nuh_layer_id=6); all parse and re-mux cleanly.

Note: Annex B input assumes PTS == DTS (no composition reordering is derived).

🤖 Generated with Claude Code

@tobbee tobbee force-pushed the feat-mvhevc-cli branch 2 times, most recently from c928ef2 to c02ef77 Compare July 4, 2026 04:14
tobbee added 5 commits July 4, 2026 07:22
Add an mp4ff-mvhevc CLI with two subcommands following the structure of the
other mp4ff tools:
- info: display the MV-HEVC structure of an mp4 (sample entries, hvcC/lhvC,
  oinf/linf sample groups, vexu/hfov spatial metadata, trgr)
- add: mux HEVC (Annex B with -fps, or an existing mp4) into a single-track
  MV-HEVC mp4 with hvcC (+lhvC for an enhancement layer), oinf/linf sample
  groups for layered input, and optional Apple spatial-video metadata
  (vexu/hfov) via -spatial

Access units are split at the first slice of each base-layer picture
(first_slice_segment_in_pic_flag), so multi-slice pictures are not split into
separate samples. Composition offsets (ctts) are carried over when re-muxing
an mp4 input. A conformant vmhd is written. As both views share one track, no
track group box is emitted.

Registered in README, .goreleaser.yml and the Makefile.
Add TestAddMultiLayerAndInfo, which builds a minimal synthetic MV-HEVC Annex B
stream (real stereo VPS with vps_extension + base/enhancement parameter sets +
placeholder slice NALUs) and runs add then info on it. This covers the
multilayer muxing path end to end: lhvC, oinf (with the base-layer operating
point and nuh_layer_id-keyed layers) and linf generation, and the info display
of them. mp4ff only manipulates boxes, so placeholder slice payloads suffice.
@tobbee tobbee force-pushed the feat-mvhevc-cli branch from c02ef77 to 966ff05 Compare July 4, 2026 05:23
@tobbee tobbee merged commit 967cabd into master Jul 7, 2026
9 checks passed
@tobbee tobbee deleted the feat-mvhevc-cli branch July 7, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant