Question: Video HW Decoder

[proconsule]

Hi there is any chance to get Video HW decoder?
I know Tegra can decode h264/hevc in hw, any resarch on getting it in libnx?
Thanks

[MRiCEQB]

Any update on this?

[Palidino]

Any plans for this ever?

[benwa]

I've started a bounty for this to be implemented.
https://app.bountysource.com/issues/116504826-question-video-hw-decoder

[averne]

I've released a FFmpeg fork enabling acceleration for all codecs supported by the hardware. I don't feel this really belongs in libnx, as most of it is not specific to the Switch/Horizon OS.
https://github.com/averne/FFmpeg/tree/tx1

[benwa]

I agree that this shouldn't be in libnx. @proconsule, can you close this so I can award @averne the bounty?

[proconsule]

I have to try this out on NXMP replacing the ffmpeg in use.

Thanks for this great result

[averne]

I have to try this out on NXMP replacing the ffmpeg in use.

Thanks for this great result

You can contact me on discord @avhe if you run into issues.

Basic integration isn't hard, but for optimal performance you would need a deko3d graphics backend for mpv. In OpenGL, you are unable to directly import frames from the decoder, so the following happens:
decoder output (block linear) -> copied to staging buffer (pitch linear) using VIC -> copied to GPU texture (block linear) using GPU copy engine.

With deko3d, you just need to create a texture with DkImageFlags_UsageVideo over the frame data.

[WinterMute]

I've released a FFmpeg fork enabling acceleration for all codecs supported by the hardware. I don't feel this really belongs in libnx, as most of it is not specific to the Switch/Horizon OS. https://github.com/averne/FFmpeg/tree/tx1

Would be awesome if you got this upstreamed to FFmpeg but also PR'd needed changes to dkP packages repo. I've put a quick test @ https://github.com/devkitPro/pacman-packages/tree/ffmpeg-update but I'm not 100% what configure options we need.

fwiw, that package can be installed using [sudo] [dkp-]pacman -U https://pkg.devkitpro.org/packages/switch-ffmpeg-6.0-1-any.pkg.tar.zst if anyone wants to test.

[averne]

Would be awesome if you got this upstreamed to FFmpeg

I contacted the FFmpeg maintainers, not sure it could be upstreamed in its current state as it contains nvidia headers (even though they are under a permissive license). There are also a number of hacks, especially surface alignment, that would need some cleaning up.

I'm not 100% what configure options we need.

At minimum, you need to add --enable-gpl since the hwaccel is gated by this. I used the following flags, but I see the devkitPro package has some others. I'll merge those flags and put together a proper PR.

--cc=aarch64-none-elf-gcc --cxx=aarch64-none-elf-g++ --ar=aarch64-none-elf-ar --nm=aarch64-none-elf-nm --extra-cflags='-isystem /opt/devkitpro/libnx/include' --enable-asm --enable-gpl --enable-tx1 --enable-static --disable-shared --enable-zlib --enable-bzlib --enable-libass --enable-libfreetype --enable-libfribidi --disable-doc --disable-programs --target-os=horizon --enable-cross-compile --cross-prefix=aarch64-none-elf- --arch=aarch64 --cpu=cortex-a57 --enable-neon --enable-pic --disable-autodetect --disable-runtime-cpudetect --disable-debug