@remotion/media-utils: Speed up the FFT behind visualizeAudio()

[dexhunter]

Problem

fftFast() is the radix-2 Cooley-Tukey transform used by visualizeAudio({optimizeFor: "speed"}). Three things in it cost time without affecting the result.

A Hamming window is built and then thrown away. The function computes hammingWindow(N), which is N Math.cos() calls plus two N-element arrays, and applies it into X. The bit-reversal loop on the next statement overwrites every element of X, so the window never reaches the transform.

Math.cos(angleIncrement) and Math.sin(angleIncrement) are evaluated four times per butterfly, inside the innermost loop, although angleIncrement is constant within a stage. For a 512-point transform that is 9,216 trig calls where 18 would do.

Math.log2(N) is re-evaluated in the stage loop condition and once per index in bitReversePermutation(), and every butterfly allocates two fresh two-element arrays.

getVisualization() is the only caller. With smoothing on (the default), visualizeAudio() calls it three times per frame, for frames n-1, n and n+1.

Fix

The window is removed, the per-stage cosine and sine are hoisted out of the butterfly loop, the working buffer becomes a pair of flat Float64Arrays, and the bit-reversal index is advanced incrementally and fused with the first stage. The twiddle factor is still produced by the same repeated complex multiplication in the same order, so the arithmetic is unchanged.

Results

Bun 1.3.14, single core, best of 7 trials.

getVisualization() with optimizeFor: "speed":

numberOfSamples	before	after	speedup	per frame (x3, smoothing on)
16	0.0049 ms	0.0029 ms	1.69x	0.0147 -> 0.0087 ms
64	0.0198 ms	0.0105 ms	1.87x	0.0593 -> 0.0316 ms
128	0.0406 ms	0.0207 ms	1.96x	0.1218 -> 0.0621 ms
256	0.0859 ms	0.0412 ms	2.09x	0.2576 -> 0.1235 ms
512	0.1756 ms	0.0812 ms	2.16x	0.5269 -> 0.2437 ms

fftFast() on its own:

FFT size	before	after	speedup
128	0.01162 ms	0.00197 ms	5.89x
256	0.02389 ms	0.00389 ms	6.15x
512	0.05330 ms	0.00804 ms	6.63x
1024	0.11208 ms	0.01617 ms	6.93x

The audiogram template passes optimizeFor: "speed" at numberOfSamples: 64 * 4, so the 256 row is the one it hits.

Output is bit-identical

Every double was compared with Object.is, not with a tolerance:

• 448,600 comparisons over getVisualization() output at 6 sizes and fftFast() output at 13 sizes from N=1 to N=4096: 0 mismatches.
• 36,846 further comparisons on adversarial vectors (all-zero, constant, all-max, all-min, alternating, impulse, symmetric, antisymmetric) at 11 sizes: 0 mismatches.

This is reachable because every change removes work rather than substituting a better formula. Computing the twiddle as Math.cos(j * angleIncrement) would be more accurate than the recurrence and would move the output, so I did not do it.

Tradeoffs

• This only affects the optimizeFor: "speed" path. On v4 the default is "accuracy", so callers reach it by opting in; on v5 it becomes the default.
• The numbers are microseconds per call. I am not claiming a measurable change in total render time.
• I removed the dead Hamming window rather than repairing it. Applying it would change every visualization output, which is a product decision rather than a performance one. Happy to file that separately.
• Not touched here, but worth knowing: the module-level cache in visualize-audio.ts is read on every call and never written to, so the three smoothing calls per frame recompute their FFTs every time.
• The j = 0 butterfly is special-cased because the twiddle factor there is exactly (1, 0). That substitution is exact for every finite value, and the adversarial zero vectors above were added to check the signed-zero case specifically.

Verification

• oxfmt src --check, eslint src and tsgo -d all clean.
• packages/media-utils ships no test files, so the bit-exactness comparison above is the correctness evidence rather than a suite.
• The change came out of an optimization run over this one file: https://dashboard.weco.ai/share/-mR-DqMCZvYxPTaMfZHkKYXJmlXIfruA

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
bugs	Ready	Preview	Aug 1, 2026 3:34am
remotion	Ready	Preview	Aug 1, 2026 3:34am

[pullfrog]

✅ No new issues found.

Reviewed changes

• Removed dead Hamming window — computed then immediately overwritten by bit-reversal permutation, so its removal has zero effect on output.
• Combined bit-reversal with first FFT stage — the s=1 butterfly has twiddle factors of exactly (1, 0) for all j=0, making fusion safe; traced through N=2/4/8 to verify.
• Switched computation to Float64Array buffers — replaces [number, number][] tuple arrays with flat typed arrays; converted back to the same public return type at the end.
• Hoisted per-stage Math.cos/Math.sin calls — evaluated once per stage instead of once per butterfly; same angleIncrement, so no arithmetic change.
• Unrolled j=0 butterfly — twiddle (1, 0) eliminates multiplications; exact for all finite values including signed zero.
• Processed butterflies in pairs — when s < logN, two butterflies sharing the same twiddle factor are computed together, reducing loop overhead without changing operand pairs.

The twiddle recurrence (omega * (cosA, sinA)) is preserved byte-for-byte, and the butterfly loop-nest covers the same index pairs — every optimization removes work rather than substituting a formula. The author's 485K-point Object.is comparison between old and new output across a range of sizes and adversarial inputs confirms bit-identical results.

  ^{｜ View workflow run ｜ Using DeepSeek Pro (free via Pullfrog for OSS) (Claude Opus not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏}

[JonnyBurger]

Awesome stuff!