-
Notifications
You must be signed in to change notification settings - Fork 0
Benchmarks
No measurements yet. This page records what to measure and what conditions to record with each result.
Shared volume. Server and worker mount the same storage. A job opens the source in place and writes renditions in place. No copies.
Object storage. The worker has no volume. It downloads the whole source before decoding, because ffmpeg wants a seekable file and a proxy ladder reads the source repeatedly, then uploads each rendition.
The second adds a transfer term the first does not have. On a large master that term may exceed the encode. CI cannot show this because fixtures are a few hundred KB.
First measurement: the ratio of transfer to encode on a realistic source. Above about 1, the fix is a design change rather than tuning, and it is the same change Cloudflare Containers require (Encoding).
Report separately. Aggregates hide which term is the problem, and the terms have different fixes.
- Fetch, encode and upload as three numbers per job.
- Realtime factor for the proxy ladder at 1080p and 4K: processing seconds per second of footage. Hardware and software separately.
- Concurrency scaling.
MEDIA_CONCURRENCYdefaults tocpus - 2per worker, and workers pull rather than being assigned, so measure whether N workers give N times throughput or contend on disk or network. - A delivery: a hundred masters arriving at once, measured from upload to last version ready. This is the number that decides usability.
- Requests and CPU-ms per review session, which Pricing needs.
- machine, core count, hardware or software encoding
- source codec, resolution, bitrate, duration
- arrangement: shared volume or object storage
- worker count and
MEDIA_CONCURRENCY - network path: same host, LAN, or public internet
scripts/integration-e2e.mjs already drives a full upload-to-ready cycle and
can kill a worker mid-encode. Timing hooks belong there rather than in a new
script that would drift from it.
30 s of 1920x1080 at 25 fps, synthetic (testsrc2), 3 vCPU in the worker
image, encoded with the settings proxies actually use: CRF 23, -g 24,
-keyint_min 24, -sc_threshold 0, audio stream-copied.
| preset | time | realtime factor | output |
|---|---|---|---|
| medium (current default) | 23.7 s | 1.3x | 19.5 MB |
| fast | 22.9 s | 1.3x | 19.6 MB |
| faster | 18.0 s | 1.7x | 19.1 MB |
| veryfast | 10.4 s | 2.9x | 17.3 MB |
| superfast | 9.5 s | 3.1x | 33.7 MB |
veryfast is 2.3x faster than medium on the same hardware. superfast buys
another 9% for double the bytes, so veryfast is the knee.
Caveat on the sizes: synthetic content compresses unlike real footage, and
here veryfast came out smaller than medium, which will not generalise. On
real material expect veryfast to be somewhat larger at the same CRF. The
timing ratio is the durable result; the size column needs redoing on a real
master.
What this implies for long sources: at medium a one-hour 1080p master is
roughly 46 minutes of encoding on 3 cores, and about 20 at veryfast. 4K is
several times worse again. Software-only encoding of long masters is slow
enough that parallelism across workers matters more than any preset choice.
20 s of 1920x1080 at 25 fps with heavy synthetic grain, encoded to CRF 12 as the reference (536 MB, about 214 Mbps). 3 vCPU. Proxy settings as above. SSIM measured against that reference.
| config | time | size | SSIM |
|---|---|---|---|
| medium @ crf23 | 110.7 s | 96.3 MB | 0.7837 |
| faster @ crf23 | 68.9 s | 64.5 MB | 0.7573 |
| veryfast @ crf23 | 51.9 s | 80.2 MB | 0.7845 |
| veryfast @ crf21 | 59.2 s | 168.6 MB | 0.8721 |
The timing is the trustworthy part: veryfast is 2.1x faster than medium here, and was 2.3x faster on the clean source. That ratio holds across two very different inputs.
The quality columns are not trustworthy, and should not be used to justify changing the default. Two reasons.
SSIM disagrees with x264 across presets. medium spent 20% more bits than
veryfast and scored slightly lower. That is what psychovisual optimisation
looks like from SSIM's point of view: medium enables stronger psy-rd,
adaptive quantisation and trellis, which preserve grain and texture that SSIM
counts as error. The metric is measuring the wrong thing for this comparison.
Settling it needs VMAF, which is not in the worker image, or eyes on real
footage.
The source is also pathological. 214 Mbps of synthetic grain is far harder than any camera master, which distorts both the sizes and the ratios between them. A 168 MB proxy for 20 seconds is 67 Mbps, which is not a proxy.
Conclusion: do not flip the default on this evidence. Ship the preset and CRF as a configurable encoding profile, keep the quality-oriented default, add libvmaf to the worker image, and settle it on real footage.
Counted by putting a logging wrapper in front of ffmpeg and running a real transcode of a 720p SDR clip with audio, which plans the usual eight renditions: proxy_1080, proxy_540, poster, sprite, waveform_data, reference_audio_1x, shuttle_audio_2x, shuttle_audio_4x.
Eight ffmpeg invocations. All eight read the source.
| # | produces |
|---|---|
| 1 | proxy_1080 + proxy_540 + sprite (the existing one-pass encode) |
| 2 | poster |
| 3 | analysis |
| 4 | reference_audio_1x |
| 5 | shuttle_audio_2x |
| 6 | shuttle_audio_4x |
| 7 | analysis |
| 8 | sprite point frames |
On a shared volume this is close to free: the file is in page cache after the first pass and the reads are local.
Streamed from object storage this costs real transfer, though less than eight reads implies -- see the measurement below, which counts bytes rather than invocations.
That is a time cost, not a money cost. R2 charges nothing for egress, and the Class B operations for eight sequential reads come to cents, so the bill is effectively unchanged. What it spends is wall-clock and bandwidth, and how much depends on the link between the worker and the bucket, which is not measured yet.
It also decides which arrangement is faster, and the answer is not the same for every worker:
- A worker with disk should download once and read locally eight times. 40 GB moved, not 320.
- A worker without room for the source -- a Cloudflare Container caps at 20 GB -- has no choice but to stream, and pays the eight reads.
So the design is to choose by whether the source fits, and folding the passes is what makes the streaming case reasonable rather than what makes it possible.
The three audio renditions are one demux apart and should be one invocation with three outputs. The poster is a single frame and belongs in the combined pass. Six of the eight look foldable into one, taking a streamed job from 320 GB to about 80 GB, and shortening every job on every arrangement.
Counting invocations overstates this. Some passes input-seek rather than read
the whole file: the poster uses -ss before -i, so it jumps to its timestamp
and decodes a short window. Only measuring bytes settles it.
A 20-second 720p source at 8 Mbps was served over local HTTP with Range support, and a full transcode was run against the URL rather than a file. Audio sidecars already folded.
| source | 19.7 MB |
| served over HTTP | 111.9 MB |
| ratio | 5.67x the source |
| HTTP requests | 18 |
| wall clock | 20.1 s |
So a 40 GB master streamed costs about 227 GB of transfer, not the 320 GB that counting reads suggested. Still worth reducing, and still time rather than money: R2 charges nothing for egress.
The more useful result is that it worked at all. ffmpeg read the source over HTTP, seeked inside it by Range, and produced every rendition correctly through the real pipeline. That is the whole adaptive-source design validated end to end before any of it was written: a worker that cannot hold a source can decode it anyway.