-
Notifications
You must be signed in to change notification settings - Fork 0
Benchmarks
No measurements yet. This page records what to measure and what conditions to record with each result.
Shared volume. Server and worker mount the same storage. A job opens the source in place and writes renditions in place. No copies.
Object storage. The worker has no volume. It downloads the whole source before decoding, because ffmpeg wants a seekable file and a proxy ladder reads the source repeatedly, then uploads each rendition.
The second adds a transfer term the first does not have. On a large master that term may exceed the encode. CI cannot show this because fixtures are a few hundred KB.
First measurement: the ratio of transfer to encode on a realistic source. Above about 1, the fix is a design change rather than tuning, and it is the same change Cloudflare Containers require (Encoding).
Report separately. Aggregates hide which term is the problem, and the terms have different fixes.
- Fetch, encode and upload as three numbers per job.
- Realtime factor for the proxy ladder at 1080p and 4K: processing seconds per second of footage. Hardware and software separately.
- Concurrency scaling.
MEDIA_CONCURRENCYdefaults tocpus - 2per worker, and workers pull rather than being assigned, so measure whether N workers give N times throughput or contend on disk or network. - A delivery: a hundred masters arriving at once, measured from upload to last version ready. This is the number that decides usability.
- Requests and CPU-ms per review session, which Pricing needs.
- machine, core count, hardware or software encoding
- source codec, resolution, bitrate, duration
- arrangement: shared volume or object storage
- worker count and
MEDIA_CONCURRENCY - network path: same host, LAN, or public internet
scripts/integration-e2e.mjs already drives a full upload-to-ready cycle and
can kill a worker mid-encode. Timing hooks belong there rather than in a new
script that would drift from it.
30 s of 1920x1080 at 25 fps, synthetic (testsrc2), 3 vCPU in the worker
image, encoded with the settings proxies actually use: CRF 23, -g 24,
-keyint_min 24, -sc_threshold 0, audio stream-copied.
| preset | time | realtime factor | output |
|---|---|---|---|
| medium (current default) | 23.7 s | 1.3x | 19.5 MB |
| fast | 22.9 s | 1.3x | 19.6 MB |
| faster | 18.0 s | 1.7x | 19.1 MB |
| veryfast | 10.4 s | 2.9x | 17.3 MB |
| superfast | 9.5 s | 3.1x | 33.7 MB |
veryfast is 2.3x faster than medium on the same hardware. superfast buys
another 9% for double the bytes, so veryfast is the knee.
Caveat on the sizes: synthetic content compresses unlike real footage, and
here veryfast came out smaller than medium, which will not generalise. On
real material expect veryfast to be somewhat larger at the same CRF. The
timing ratio is the durable result; the size column needs redoing on a real
master.
What this implies for long sources: at medium a one-hour 1080p master is
roughly 46 minutes of encoding on 3 cores, and about 20 at veryfast. 4K is
several times worse again. Software-only encoding of long masters is slow
enough that parallelism across workers matters more than any preset choice.
20 s of 1920x1080 at 25 fps with heavy synthetic grain, encoded to CRF 12 as the reference (536 MB, about 214 Mbps). 3 vCPU. Proxy settings as above. SSIM measured against that reference.
| config | time | size | SSIM |
|---|---|---|---|
| medium @ crf23 | 110.7 s | 96.3 MB | 0.7837 |
| faster @ crf23 | 68.9 s | 64.5 MB | 0.7573 |
| veryfast @ crf23 | 51.9 s | 80.2 MB | 0.7845 |
| veryfast @ crf21 | 59.2 s | 168.6 MB | 0.8721 |
The timing is the trustworthy part: veryfast is 2.1x faster than medium here, and was 2.3x faster on the clean source. That ratio holds across two very different inputs.
The quality columns are not trustworthy, and should not be used to justify changing the default. Two reasons.
SSIM disagrees with x264 across presets. medium spent 20% more bits than
veryfast and scored slightly lower. That is what psychovisual optimisation
looks like from SSIM's point of view: medium enables stronger psy-rd,
adaptive quantisation and trellis, which preserve grain and texture that SSIM
counts as error. The metric is measuring the wrong thing for this comparison.
Settling it needs VMAF, which is not in the worker image, or eyes on real
footage.
The source is also pathological. 214 Mbps of synthetic grain is far harder than any camera master, which distorts both the sizes and the ratios between them. A 168 MB proxy for 20 seconds is 67 Mbps, which is not a proxy.
Conclusion: do not flip the default on this evidence. Ship the preset and CRF as a configurable encoding profile, keep the quality-oriented default, add libvmaf to the worker image, and settle it on real footage.
Counted by putting a logging wrapper in front of ffmpeg and running a real transcode of a 720p SDR clip with audio, which plans the usual eight renditions: proxy_1080, proxy_540, poster, sprite, waveform_data, reference_audio_1x, shuttle_audio_2x, shuttle_audio_4x.
Eight ffmpeg invocations. All eight read the source.
| # | produces |
|---|---|
| 1 | proxy_1080 + proxy_540 + sprite (the existing one-pass encode) |
| 2 | poster |
| 3 | analysis |
| 4 | reference_audio_1x |
| 5 | shuttle_audio_2x |
| 6 | shuttle_audio_4x |
| 7 | analysis |
| 8 | sprite point frames |
On a shared volume this is close to free: the file is in page cache after the first pass and the reads are local.
Streaming from object storage it is not. A 40 GB master read eight times is 320 GB moved per job. That makes folding the remaining passes a prerequisite for streaming rather than an optimisation to do afterwards, which is the opposite of the order it looks like it should be done in.
The three audio renditions are one demux apart and should be one invocation with three outputs. The poster is a single frame and belongs in the combined pass. Six of the eight look foldable into one, which would take a streamed job from 320 GB to about 80 GB.