6.25.0 - A source is measured, not taken at its word

Drop-in from 6.24.0. Two black-screen classes on the loopback VOD path, no source-breaking API change. Both reported on 6.21.0 by @RomanLiberda, in #365 and #366, against full UHD-BD remuxes.

Neither turned out to be what the reports said, and in both cases the false lead was a reasonable one.

The framing of a source was decided from its description (#365)

A Dolby Vision Profile 7 MKV reached AVPlayer as -11855 Cannot Decode, with the video track reporting fourCC=<no fdesc>. The stated cause was that the generated hvcC carries no parameter-set arrays, the class fixed for DV P5 MP4 in 2.0.2.

The hvcC is fine. Three fixtures built from one source (proper record, Annex-B record, Annex-B record plus Annex-B blocks) produce a byte-identical, correct 116-byte hvcC in all three cases. And fourCC=<no fdesc> does not say what it looks like it says: the engine logs it from item.tracks on an item that has already failed, where the AC-3 track prints exactly the same thing.

What the report's own log does pin down is the source's real anomaly, in one line: extradata=726B head=00000140010c01ff. That config record is Annex B, not an hvcC, which is the shape a Matroska remux has when its CodecPrivate is Annex B or missing entirely (libavformat then synthesises Annex-B extradata from the first in-band parameter sets).

From there it is the mp4 muxer's own rule that does the damage. mov_write_packet_internal decides whether to convert samples by looking at the extradata, not at the packet: "extradata is Annex B, assume the bitstream is too and convert it". On a source whose packets are in fact length-prefixed, ff_hevc_annexb2mp4 then runs over MP4-framed samples and finds start codes only where a 4-byte NAL length happens to read as 00 00 01. Measured on a 1080p fixture: a 2,158,448 B segment came out at 61,912 B, the init.mp4 stayed byte-for-byte valid, and AVPlayer reached readyToPlay without ever producing a frame.

The engine now measures the framing on real packets at open (does a length-prefixed walk consume the packet exactly?) and converts the config record to an hvcC when the two disagree, so the muxer's own test comes out right. The head bytes are deliberately not the discriminator: every NAL of 256 to 511 bytes carries the length prefix 00 00 01 xx, and SEI or parameter-set NALs sit in that band routinely.

The measured framing is then handed to every NAL walker in the session, which repaired a second, quieter defect on the same sources. The DV P7 to 8.1 RPU rewrite assumed length prefixes, so on Annex-B packets it read 00 00 01 40 as a 320-byte NAL, found no RPU, reported success, and shipped the P7 RPU and the enhancement layer inside a container the muxer had already rewritten to 8.1: the mixed-profile hazard #135 warns about, reached silently. It now takes the framing and emits the packet in the framing it received.

The in-band parameter-set rebuild from #19 no longer runs on Annex-B extradata either. Bytes 21 and 22 of an Annex-B HEVC record pass its two checks by construction rather than by luck, because the 00 00 03 emulation-prevention pattern in a Main10 VPS sits exactly there, so byte 21 reads as naluLengthSize 4 and byte 22 as numOfArrays 0. canonicalizeHEVCConfigRecord has always had the configurationVersion guard; that path never did.

A search bounded in bytes, on a source measured in seconds (#366)

A second remux died with muxerFailed and a permanent black screen, because the selected audio track was a sparsely interleaved legacy dub whose first packet lies far past the start.

The first segment of an AC-3 / E-AC-3 source cannot be cut until one parsed audio packet has reached the muxer (#222), and the search for that packet read forward from wherever the pump stopped, bounded at 128 MiB. That bound is a byte bound, so what it buys shrinks as the bitrate grows: five minutes of a 3 Mbps encode, ten seconds of a 97 Mbps UHD one. No byte budget fixes a track whose first packet sits hundreds of MiB in.

It never had to be the first frame. AC-3 and E-AC-3 are one complete syncframe per packet, movenc builds the whole sample entry from whichever frame it gets, and the prime frame's timestamp is discarded anyway (the primed fragment is truncated and frag_discont re-armed). So when the forward scan comes back empty, the pump now seeks to four positions, midpoint first, and takes any frame the track yields there. Nothing in the container could have pointed at the track instead: on a fixture whose first audio packet sits at 211 MiB, that track's AVStream.start_time still reads 0.

On that fixture the forward scan reads its 128 MiB and finds nothing, the midpoint probe reads 32 MiB and finds nothing, and the 90 % probe captures a 418 B frame after two packets. The session then plays, the init.mp4 carries a real dac3, and the audio lands exactly where the source put it: segment 19 holds video 76.000 to 79.958 plus the source's first audio packet at 79.994, segment 20 holds audio 80.028 to 83.965. The prime taken from 84 s did not drag the timeline with it.

Falling back to a different audio track was considered and not built: with the prime found, the track the viewer selected is the one that plays.

An unmuxable source now says so

The reason #366 reached a viewer as a black screen rather than an error is separate from the search. The exhausted arm of the VOD muxer-failure revive was a bare return: no producer, no restart, no error, so the provider answered 404 init.mp4 empty forever while AVPlayer sat in waitingToPlay. Its sibling arm for read errors, 150 lines up the same file, has surfaced its own exhaustion since #169.

Measured on a source whose audio track carries no blocks at all: before, state=playing phase=rebuffering for the whole session; after, state=error("Source audio cannot be muxed (code -22)") inside a second.

The terminal failure now carries a reason as well as a code, which fixed three existing call sites in passing: the #358 unproducible segment and the sequential-origin reposition were both reporting a failed read for something that read fine.

The structural verdict is recorded on the session, so the three revive attempts do not each pay the full search again (about 256 MiB of reads per attempt) to reach the same answer. It is deliberately not recorded when a read threw or the pump was stopped: those say nothing about the track, and treating them as final would turn a transient I/O hiccup into a dead session.

API

VideoNALFraming is new and public (.annexB / .lengthPrefixed(size:)). DoviRpuConverter.convertPacketToProfile81 and enhancementLayerType take it as a defaulted parameter, so existing calls compile and behave unchanged; a caller that hands them Annex-B packets now has a way to say so. Nothing else changed, and hosts need no change.

Verifying it

No muxer will write the file shape behind #365: matroskaenc always converts the config record with ff_isom_write_hvcc and always reformats Annex-B packets to length prefixes. It has to be patched into a file that already exists, so the generator is in the repo:

ffmpeg -i src.mp4 -c copy base.mkv
python3 Scripts/mkv-annexb-fixture.py base.mkv annexb-record.mkv --annexb-codecprivate
python3 Scripts/mkv-annexb-fixture.py base.mkv annexb-both.mkv --annexb-codecprivate --annexb-blocks

Both edits are byte-count preserving (the config record is padded back to length with an EBML Void, and a 4-byte length prefix is the same width as a start code), so no parent element size has to be rewritten. The three files are each other's control: after the fix all three produce a byte-identical init.mp4 and a byte-identical 2,158,448 B seg0 through aetherctl serve, and all three reach rfd=y in a real AVPlayer.

The #366 fixture needs no patching, only an offset input:

ffmpeg -f lavfi -i "testsrc2=size=1920x1080:rate=24" -t 95 -c:v libx264 -b:v 22M late-video.mp4
ffmpeg -i late-video.mp4 -itsoffset 80 -f lavfi -i "sine=duration=14" \
  -map 0:v -map 1:a -c:v copy -c:a ac3 late-audio.mkv

Its first audio packet sits at byte 221,172,397. Cutting the same file with -t 60 -c copy leaves the AC-3 track declared with zero blocks, which is the source the terminal arm is measured against.

1824 swift-testing plus 504 XCTest green, tvOS build clean.