feat: add mp4ff-mvhevc command line tool#515
Merged
Merged
Conversation
c928ef2 to
c02ef77
Compare
Add an mp4ff-mvhevc CLI with two subcommands following the structure of the other mp4ff tools: - info: display the MV-HEVC structure of an mp4 (sample entries, hvcC/lhvC, oinf/linf sample groups, vexu/hfov spatial metadata, trgr) - add: mux HEVC (Annex B with -fps, or an existing mp4) into a single-track MV-HEVC mp4 with hvcC (+lhvC for an enhancement layer), oinf/linf sample groups for layered input, and optional Apple spatial-video metadata (vexu/hfov) via -spatial Access units are split at the first slice of each base-layer picture (first_slice_segment_in_pic_flag), so multi-slice pictures are not split into separate samples. Composition offsets (ctts) are carried over when re-muxing an mp4 input. A conformant vmhd is written. As both views share one track, no track group box is emitted. Registered in README, .goreleaser.yml and the Makefile.
Add TestAddMultiLayerAndInfo, which builds a minimal synthetic MV-HEVC Annex B stream (real stereo VPS with vps_extension + base/enhancement parameter sets + placeholder slice NALUs) and runs add then info on it. This covers the multilayer muxing path end to end: lhvC, oinf (with the base-layer operating point and nuh_layer_id-keyed layers) and linf generation, and the info display of them. mp4ff only manipulates boxes, so placeholder slice payloads suffice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The capstone of the MV-HEVC effort: a command line tool that ties the VPS parser,
lhvC,oinf/linfbuilders, sample entries and spatial-video boxes together.mp4ff-mvhevchas two subcommands, structured like the othermp4ff-*tools (main→run(os.Args, os.Stdout), anoptionsstruct +parseOptionsper subcommand,internal.GetVersion()):info <input.mp4>— displays the MV-HEVC structure: sample entries,hvcC/lhvC, Dolby Vision configuration (dvcC/dvvC/dvwC),oinf/linfsample groups,vexu/hfovspatial metadata, andtrgr(including unregistered track-group types).add [options] <input.hevc|mp4> <output.mp4>— muxes HEVC into a single-track MV-HEVC mp4:-fps) or an existing HEVC/MV-HEVC mp4 as input.hvc1withhvcC(andlhvCwhen an enhancement layer is present).oinf/linfsgpd+sbgpfor layered (multi-view) input.vexu/hfov) via-spatial(with-baseline,-hfov,-hero,-reversed).Registered in the README,
.goreleaser.ymland the Makefile.Correctness (adapted from the reference implementation)
A focused review of the muxer surfaced several issues that are fixed here:
first_slice_segment_in_pic_flag), so a multi-slice picture is no longer split across multiple samples.ctts(composition offsets / B-frame reordering) is carried over when re-muxing an mp4 input — verified byte-identical to the source onmp4/testdata/ed_hevc.mp4.vmhd(flags = 1) is written viamp4.CreateVmhd().hvcC(suffix SEI is no longer mislabeled as prefix).stcooffsets fails cleanly (co64 is not emitted).cstgtrack group the reference writes is not written.cstgis the complete subset track grouping of ISO/IEC 14496-15 §9.5.1 (GPAC writes it withtrack_group_id = 1000 + track_idwhen muxing layered HEVC, which is where the reference's hardcoded 1001 comes from), but it marks the minimal set of multiple tracks that together carry the complete L-HEVC bitstream — for the single-track output of this tool it conveys nothing, and Apple's own muxer writes no track group either. Thecstgtype is now registered onTrackGroupTypeBox(alongsidemsrc/ster), soinfodecodes and displays it in files that have it.Tests
Unit tests for the helpers, plus end-to-end
add→inforound-trips:-spatial(asserts thevexu/hfov/strioutput) and an mp4 re-mux (assertshvc1and thatcttsis carried over).vps_extension, base + enhancement parameter sets, placeholder slice payloads — mp4ff only touches boxes) is muxed and asserted to producelhvC+oinf/linf.cmd/mp4ff-mvhevc/testdata/stereo_spatial.mp4(4.9 KB): a genuine MV-HEVC spatial video encoded with Apple VideoToolbox from synthetic gradient frames by the checked-in generatortestdata/gen_stereo_spatial.swift(entirely own content, so redistributable; ffmpeg confirmsview_ids_available=0,1). Since it is muxed by an independent implementation, it testsinfoagainst externally-produced boxes (hvcC+lhvC,vexuwithstri/hero/blin,hfov,ctts, and nooinf/linf), the-idrflag, and drives the multilayer mp4-input re-mux path end-to-end: parameter sets picked up fromhvcC+lhvC,oinf/linfbuilt from a real VideoToolbox VPS, andcttscarry-over.dvwCbox and atrgrwith registered (ster,cstg) and unregistered group types are attached in-test and asserted in theinfooutput;cstground-trip is covered inmp4/trgr_test.go.go test ./...,go vet,golangci-lintclean.The tool has also been validated locally against MV-HEVC files that cannot be checked in (Apple/GPAC-produced spatial videos, Dolby Vision profile 20, and a stream with enhancement-layer
nuh_layer_id=6); all parse and re-mux cleanly.Note: Annex B input assumes PTS == DTS (no composition reordering is derived).
🤖 Generated with Claude Code