v0.12.0-vk-video-decode

New

Hardware accelerated video decoding (H.264, for now) using Vulkan Video Decode.

Island's new le_video_decoder module allows you to use your GPU most efficiently by shifting video processing onto dedicated video decode hardware. Instead of using an extra dependency, le_video_decoder implements this by using the new Vulkan Video API, therefore keeping video decoding perfectly on the GPU.

The new video decoder module provides a simple API that aims to abstract away any gnarly sync issues: the latest decoded frame is made available as a plain image resource handle which synchronizes nicely and implicitly with Island's rendergraph. By default, videos will loop perfectly; YCrCb conversion happens automatically using specialized Vulkan samplers. A new example application shows the video player in action.

Adding and implementing Vulkan Video Decode has been a long time in the making; This release has been tested on Linux (ubuntu 22.04 LTS) and Windows (11) and an Nvidia 4090 GPU, where I got to comfortably decode and play 14 1920x1080@50fps videos simultaneously.

Other GPUs will most likely work, too as long as they support a Video Decode Queue (it's recommended to update drivers, as Vulkan Video is still fairly new). I'd love to find out how this fares on Intel and/or AMD GPUs - test reports, PRs or hardware donations (so I can test & implement fixes) are most welcome ;)

There are a lot more features and bug fixes in this release.

What's Changed

• NEW Video decoder by @tgfrerer in #57
• e383df6 NEW Video player example app: an Island example application that shows how to use the new video decoder module
• 9704b25 NEW implement automatic hardware-accelerated YCbCr conversion for textures annotated with __ycbcr__ in sampler code
• b4210f4 NEW add callback forwarding for Windows, so that callbacks can be hot-patched on all supported architectures now - one-file minimal POC here
• d7ec70d Refactor TWEAKS: refactor near-instant code-tweak mechanism by putting it into its own module, le_tweaks -- see le_tweaks for usage hints
• Update Vulkan struct scaffold generator helper: now even more interactive, and useful for Vulkan Video structs, too.
• Update VOLK to version 250
• Fix querying correct device properties from device list by @ldkuba in #55
• Fix missing standard includes by @jopadan in #58

New Contributors - Thank you :)

• @ldkuba made their first contribution in #55
• @jopadan made their first contribution in #58

Full Changelog: v0.11.0-tracy...v0.12.0-vk-video-decode