iPad Safari: WebGPU inference drops 3x after ~10 s of sustained load; batching frames fixed it

[Tane van der Boon (pullmyleg)]

I was testing a small real-time object detector in Safari on an iPad (A16, iPadOS 26.6) at home with
onnxruntime-web 1.30 and the WebGPU EP, and hit a pattern that took a while to pin down. Posting the
measurements and the fix in case it saves someone else the detour.

Setup: a ~5 MB fp16 YOLO-style detector (static 640x640 input, 1x84x8400 output, a couple of hundred
kernels), run in a Worker on a fixed image so the camera was out of the picture.

Symptom: 57 ms per run (14 fps) for the first 10 s, then a hard step to 185 ms (4.7 fps) that stays
for as long as you keep running. A 30 s pause restores 57 ms; it then degrades again after another
10 s. Recreating the session or the worker restores it only for a few seconds.

What it is not (each tested on the device, 2 minutes per run):

• Not the ORT bundle: onnxruntime-web/webgpu (asyncify), /jspi (iPadOS 26.6 has JSPI) and the
  JSEP bundle all show the same step. The JSEP bundle is just slower from the start (138 ms).
• Not the wasm JIT: padding large wasm functions past JSC's OMG size cap changed nothing.
• Not CPU load: a pure JS worker at 100% duty for 2 minutes stays flat.
• Not GPU load on its own: a WebGPU compute shader at 100% duty for 2 minutes stays flat.
• Not input size: 640, 480 and 416 all cost 52 to 57 ms per run when fresh.
• Not CPU preprocessing or dispatch overhead: letterboxing in a compute shader into a GPU buffer
  (Tensor.fromGpuBuffer, preferredOutputLocation: 'gpu-buffer') with enableGraphCapture gave the
  same 54 ms and the same step.

So it looks like the SoC moving the GPU to a low-power state once the CPU plus GPU load from the
inference loop stays high for about 10 s. Single-frame runs were sustainable only up to 8 a second
(9 tripped after 30 s, 10 and 15 after 10 s).

The fix: because the per-run cost barely depends on how many pixels you feed it, I exported the
model with a static batch of 4 and run four captured frames per session.run. On the iPad that costs
198 ms per run, so 4 runs a second gives 15 frames a second, held flat for 5 minutes. At 480 input
batch 4 gives 18 frames a second. Result latency is up to ~0.4 s after capture, which is fine for
tracking. Mac Chrome for comparison: 20 ms for 1 frame, 46 ms for 4.

Practical notes:

• Read the batch size from session.inputMetadata[0].shape[0] rather than config so the model and
  the pump cannot disagree.
• The frames go to the GPU with queue.copyExternalImageToTexture from an ImageBitmap and a small
  compute shader writes the letterboxed CHW fp16 planes (pack2x16float) into one buffer that is
  wrapped with Tensor.fromGpuBuffer once and reused.
• If you benchmark this on a Mac, keep Safari frontmost. When it is not, macOS drops the WebContent
  process to background priority and a JS worker slows 9x, which looks exactly like the iPad problem
  and is not.

Happy to share more numbers if useful.