VideoEncoder performance with hardware acceleration

[dustinkerstein]

Today I tried to put together a streaming transcoder with WebCodecs. It's functional, and was surprisingly very easy to build, but I am running into some performance issues (and for my use-case, I can't drop frames). To isolate things a bit, I put together a very simple JSFiddle that focuses solely on the VideoEncoder portion of the code. You can find the demo here - https://jsfiddle.net/badru54k/

On my 2018 Macbook Pro, I can only maintain around 25fps. This feels a bit low to me considering it is using the hardware encoder (verified in the Chrome Media tab). With FFmpeg + VideoToolbox, I can encode 3840x2160 h264 at around 65-70fps. Is this amount of overhead to be expected, or is there maybe something else going on?

Thanks,
Dustin

PS - I can also replicate the same behavior in Edge and in Opera (with slightly worse performance).

[aboba]

One thing to keep in mind is that support for hw acceleration can vary with width/height/framerate, etc. So if you're expecting to be able to encode 3840 x 2160 h264 at 65 fps with a 2018 MacBook Pro, you might want to check first whether this can be supported via isConfigSupported. MediaCapabilities API may also provide useful info on what you can expect (e.g. power efficient, smooth, etc.). A basic MC demo is here.

[dalecurtis]

This seems likely to be an implementation specific issue. Can you file a https://crbug.com/new issue instead? This repo is for issues with the spec and samples. It's fine to file issues here if you're unsure where it belongs though.

[dustinkerstein]

Thanks for the replies. Yah, it definitely could be an implementation specific issue, and I'll definitely post an issue at the Chromium tracker. But I'm still a little curious - as I imagine WebCodecs is a fairly lightweight wrapper on VideoToolbox (for MacOS), I would expect they would perform similarly. Maybe there's a better way for me to test. If you happen to know offhand, what would be the most performant type of VideoFrame to encode? Ie. CanvasImageSource vs. ArrayBuffer, alpha, pixel format, etc.

Upon another quick test with that JSFiddle (which isn't using isConfigSupported for simplicity) shows that my max encode rate at 1080p is ~80fps, whereas with FFmpeg + VideoToolbox I hit ~280fps. Thanks again!

[dustinkerstein]

FYI, here is the new Chromium issue - https://bugs.chromium.org/p/chromium/issues/detail?id=1326570