Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): hardware decoding, libplacebo for tone-mapping #9402

Closed
wants to merge 10 commits into from

Conversation

mertalev
Copy link
Contributor

@mertalev mertalev commented May 12, 2024

Description

Edit: Reducing the scope of this PR to only change the process for NVENC and possibly QSV and VAAPI depending on Vulkan support. This is because Vulkan is quite new in FFmpeg and we would need a more bleeding edge version to use it more extensively. For the time-being, this does unfortunately mean we essentially have 3 ways of tone-mapping: zscale (CPU), Vulkan (NVENC and maybe QSV and VAAPI) and OpenCL (RKMPP).

This PR overhauls the decoding and tone-mapping behavior for hardware-accelerated transcoding:

  • Vulkan-based pipeline shared across all devices, including CPU
    • CPU on Vulkan works, but with a roughly %30 slowdown compared to the existing pipeline, likely overhead from the upload/download process. So the existing pipeline will still be used for CPU.
    • Decoding is still done through device-specific APIs, but can be done through Vulkan as well in the future
  • Codec and API-specific logic is reduced to encoding
  • libplacebo (used in mpv) allows for feature-rich and zero-copy hardware tone-mapping
  • All backends have access to the same set of tone-mapping options

How Has This Been Tested?

Tested on CPU and NVENC for both transcoding and video thumbnails. Keeping this in draft until it's been tested with the other APIs.

Fixes #4321
Fixes #8411

Copy link

cloudflare-workers-and-pages bot commented May 12, 2024

Deploying immich with  Cloudflare Pages  Cloudflare Pages

Latest commit: 9ffc2e5
Status:🚫  Build failed.

View logs

@mertalev mertalev force-pushed the feat/server-hw-decoding branch 2 times, most recently from 4eeb279 to 9ffc2e5 Compare May 13, 2024 03:00
@mertalev
Copy link
Contributor Author

Closing in favor of #9452

@mertalev mertalev closed this May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant