Narrowed after measurement. This issue originally asked where the VideoToolbox/software decode crossover sits for 4K, 10-bit and HEVC. 4K is now answered and is no longer part of the question.
4K — measured, software wins
Decode only, 1200 frames, best of three passes, Mac mini M1, with the 1080p case kept as a control against the end-to-end figure in #583:
| source |
software |
VideoToolbox |
ratio |
| 1080p (control) |
2586 fps |
212 fps |
12.19× |
| 4K |
849 fps |
71 fps |
11.88× |
The ratio barely moves with resolution: VideoToolbox's cost is a fixed per-frame latency, so four times the pixels does not rescue it. 4K is in fact where hardware decode hurts most — 71 fps is below real time for a 4K60 timeline.
Recorded in Decoder::open_with; no resolution threshold exists in the condition, deliberately.
What is still open
codec_id == H264 && format == YUV420P excludes both of these by construction, so they keep VideoToolbox untested:
- HEVC 8-bit. iPhone footage and some screen captures. The software HEVC decoder is materially more expensive than H.264's, so the ratio could genuinely invert here — this is the case most likely to have a real crossover.
- H.264 / HEVC 10-bit. Rarer, and it needs work beyond the predicate:
mac_frames::CpuFrames converts to 8-bit NV12 (AV_PIX_FMT_NV12, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange), so routing 10-bit through the software path would silently truncate to 8 bits. The predicate is what currently prevents that, which is worth knowing before anyone widens it.
How to answer them
No code change is needed to measure either: OPENSCREEN_MAC_DECODE=videotoolbox|software forces both sides on the same binary. The cheap decode-only method above was validated against the full export by its 1080p control, so it is enough to find the direction; confirm anything surprising with a full export A/B against a per-cycle ffmpeg floor.
Also worth a thought while doing it: the 4K figures were taken on an upscaled screen recording, which is easy to decode for its pixel count. A denser 4K bitstream is where the software decoder's cost actually scales and VideoToolbox's does not, and it is the fair attempt at overturning the result above.
Narrowed after measurement. This issue originally asked where the VideoToolbox/software decode crossover sits for 4K, 10-bit and HEVC. 4K is now answered and is no longer part of the question.
4K — measured, software wins
Decode only, 1200 frames, best of three passes, Mac mini M1, with the 1080p case kept as a control against the end-to-end figure in #583:
The ratio barely moves with resolution: VideoToolbox's cost is a fixed per-frame latency, so four times the pixels does not rescue it. 4K is in fact where hardware decode hurts most — 71 fps is below real time for a 4K60 timeline.
Recorded in
Decoder::open_with; no resolution threshold exists in the condition, deliberately.What is still open
codec_id == H264 && format == YUV420Pexcludes both of these by construction, so they keep VideoToolbox untested:mac_frames::CpuFramesconverts to 8-bit NV12 (AV_PIX_FMT_NV12,kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange), so routing 10-bit through the software path would silently truncate to 8 bits. The predicate is what currently prevents that, which is worth knowing before anyone widens it.How to answer them
No code change is needed to measure either:
OPENSCREEN_MAC_DECODE=videotoolbox|softwareforces both sides on the same binary. The cheap decode-only method above was validated against the full export by its 1080p control, so it is enough to find the direction; confirm anything surprising with a full export A/B against a per-cycle ffmpeg floor.Also worth a thought while doing it: the 4K figures were taken on an upscaled screen recording, which is easy to decode for its pixel count. A denser 4K bitstream is where the software decoder's cost actually scales and VideoToolbox's does not, and it is the fair attempt at overturning the result above.