Skip to content

6.14.0 - Sidecar subtitles that survive leaving the screen

Choose a tag to compare

View all tags
@superuser404notfound superuser404notfound released this 07 Aug 16:31
· 928 commits to main since this release
6.14.0
56c17a0

Two changes on the nativeRemoteHLS bypass, both about LoadOptions.externalSubtitles. Drop-in from 6.13.0, no source change on the consumer side.

Fixed: the declaration was dropped, silently

load() routes a nativeRemoteHLS session into loadRemoteHLS and returns from there, which is well before the point where the probe path registers LoadOptions.externalSubtitles. A host that declared sidecars on a remote HLS source got an empty subtitleTracks back: no error, no log line, and nothing that distinguishes a dropped option from a source that has no subtitles at all. The AE#154 reroute onto the same bypass dropped them too.

The legible-group discovery would have undone the registration in any case. It assigned subtitleTracks wholesale rather than merging, so on a source carrying its own renditions the sidecars would have been delisted a beat after they appeared. Both branches now register, and the discovery merges.

Added: those sidecars can become real renditions

The overlay is not drawn once the picture leaves the host's view hierarchy. PiP, AirPlay and a wired external display all lose the subtitle, and for a Plex or Jellyfin transcode served as .m3u8 and told subtitles=none, the sidecar is the only copy of the text that exists.

There is no API route to fix that. Media selection on an HLS asset comes from the playlist and from nowhere else; no AVMutableMediaSelectionGroup can be attached to a remote AVURLAsset. So addExternalSubtitleTrack is overlay-only by construction, and the only way to a rendition is a master the engine writes itself.

For a VOD source it now does. The engine fetches the origin master, makes every variant, audio, I-frame and key URI absolute against it, adds one EXT-X-MEDIA:TYPE=SUBTITLES per text sidecar, and serves that master from the loopback origin:

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="DE",LANGUAGE="de",DEFAULT=NO,AUTOSELECT=NO,URI="subs_0.m3u8"
#EXT-X-STREAM-INF:RESOLUTION=1920x1080,CODECS="hvc1.2.4.H153.B0,mp4a.40.2",...,SUBTITLES="subs"
https://devstreaming-cdn.apple.com/.../cmaf/hvc/1080p_5800/hvc_1080p_5800.m3u8

The media never moves. AVPlayer fetches every A/V byte straight from the origin, which is the property this bypass exists for, so E-AC-3 / Atmos passthrough is untouched. Only the master and the WebVTT renditions are local.

Details that decided the shape:

  • A variant references exactly one SUBTITLES group, so the injected renditions join the origin's own group when it has one, and every distinct group when it declares several. A variant referencing none is pointed at the engine's.
  • NAME must be unique within a group or AVFoundation collapses the options into one, so a sidecar repeating an origin name gets a numeric disambiguator.
  • DEFAULT=NO,AUTOSELECT=NO, no FORCED: the same discipline as the loopback master. Nothing self-engages; the host that declared the sidecar decides when it plays.
  • The track keeps the external id it was registered under, but selecting it drives AVMediaSelection instead of the sidecar decode. Running both would draw the same cues twice, and only the rendition survives leaving the view hierarchy.
  • Rendition URIs stay relative, so the AirPlay swap to the device's LAN IP carries them along.

Anything that does not line up keeps the origin URL and overlay-only subtitles, with one log line naming the reason: a live playlist (no EXT-X-ENDLIST), a bitmap sidecar, a playlist that will not rewrite, an origin slower than the 5 s budget. A load is never failed over a subtitle feature.

displayName is not the rendition's NAME

Worth recording, because the first implementation of the selection lookup assumed otherwise and matched nothing. AVMediaSelectionOption.displayName is a localized language name derived from LANGUAGE; the NAME attribute is discarded. Measured against Apple's own CMAF master:

opt0 display=German     lang=de common=m3u8/NAME=DE
opt2 display=Chinese    lang=zh common=m3u8/NAME=简体中文
opt5 display=French     lang=fr common=m3u8/NAME=Français

The verbatim attribute survives in commonMetadata under the m3u8/NAME identifier. That is what both the selection lookup and the deduplication key on now.

Verified

Full suite 1595/1595, tvOS and iOS Simulator builds green. End to end against that CMAF master with an .srt sidecar: 8 legible renditions surfaced (7 origin, 1 injected, no duplicate entry for the sidecar), HEVC 1080p playing from the CDN throughout, and selecting external id 100000 fetches subs_0.m3u8 and subs_0_0.vtt from the loopback origin.

aetherctl play --sidecar <lang>=<path> is new, so the whole chain is reproducible without a device, and the run summary now prints the settled subtitle track list and the active selection.

Reported in #316.

Assets 3
Loading