-
-
Notifications
You must be signed in to change notification settings - Fork 13k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oneVPL for intel gpus #264621
oneVPL for intel gpus #264621
Conversation
currently building/running the passthru tests for ffmpeg-full which is taking a long time on my machine (x86_64-linux), but trying to be as thorough as I can. I'll run nixpkgs-review next |
I think it's problematic to completely remove Media SDK because that means dropping support for CPUs older than 3 years. My preference would be to have oneVPL dispatch to Media SDK and make the Media SDK derivation private to oneVPL. Intel mentions an unspecified security issue with Media SDK in their deprecation notice, so we could make an option that allows security conscious users to disable Media SDK dispatch. |
ffmpeg is the only package in Nixpkgs that uses Media SDK, so I'm assuming that removing Media SDK from the top level package set would largely go unnoticed as long as ffmpeg could dispatch to it. |
Here's a log for the build failure of As media SDK is only needed to support x86 CPUs we should only have oneVPL depend on it on x86_64. oneVPL itself should also be packaged for aarch64 as it may be used with intel dedicated graphics. |
c8635bd
to
fb85676
Compare
the cause of onevpl-intel-gpu being broken on aarch64-linux is this part of upstream's cmake: add_library(mfx_require_sse4_properties INTERFACE)
if (CMAKE_C_COMPILER_ID MATCHES Intel)
target_compile_options(mfx_require_sse4_properties
INTERFACE
$<$<PLATFORM_ID:Windows>: /arch:sse4.2>
$<$<PLATFORM_ID:Linux>: -xsse4.2>
)
else()
target_compile_options(mfx_require_sse4_properties
INTERFACE
# on Windows MSVC SSE4.2 supported by default
$<$<PLATFORM_ID:Linux>: -msse4.2>
)
endif() which goes beyond my cmake ability to patch. I agree that aarch64-linux hosts should be able to use this intel gpu driver, but obviously intel has been narrow-minded in their implementation as of now :) I marked this driver as broken for aarch64 for now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ffmpeg part LGTM
fb85676
to
a57ae1e
Compare
I think that's a great next step but this PR is "big" enough to me (not in lines changed, just in stepping out of my comfort zone with packages that affect lots of users). I can try a draft of that change after if that's alright |
Note that, while you are expected to do reasonable QA on your own, your PR does not have to be perfect. If we find out later that this breaks something unforseen, we can always revert it again (or otherwise fix it). That's one of the purposes of the unstable channel. Obviously we'd prefer not to have to do that but it wouldn't be the end of the world either. |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
Are there any updates on this? I would love to switch my desktop to NixOS but need to encode av1 with my GPU. Or can I contribute something? |
yeah, it seems this died after a month.. |
Feel free to use this as it is, it is working for me :) You can take this branch as an extra flake input and apply an overlay for just the packages I added or changed. If they need to be updated just let me know |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs a rebase for ffmpeg 6.1.
# For forward compatibility with oneVPL. | ||
# To ensure the VPL backend is dispatched at runtime, export INTEL_MEDIA_RUNTIME=ONEVPL | ||
# See https://github.com/Intel-Media-SDK/MediaSDK#media-sdk-support-matrix | ||
runtimeInputs = [ oneVPL ]; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means it still works on older GPUs but it uses oneVPL on newer ones, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it only uses oneVPL at runtime if the environment variable INTEL_MEDIA_RUNTIME
is ONEVPL
a57ae1e
to
1864e80
Compare
I've made a PR to upstream the libvpl patch here: #291559 |
Co-authored-by: Evan Richter <evanjrichter@gmail.com>
030a8a4
to
88add7e
Compare
@@ -92,6 +92,7 @@ | |||
, withVmaf ? withFullDeps && !stdenv.isAarch64 && lib.versionAtLeast version "5" # Netflix's VMAF (Video Multi-Method Assessment Fusion) | |||
, withVoAmrwbenc ? withFullDeps && withVersion3 # AMR-WB encoder | |||
, withVorbis ? withHeadlessDeps # Vorbis de/encoding, native encoder exists | |||
, withVpl ? false # Hardware acceleration via intel libvpl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How much extra closure size would this be if you enabled it now that the driver bit is impure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
showing just the different lines:
# withMfx
/nix/store/556zw392nqnc0r8fg9n7yid1w920zd80-intel-media-sdk-23.2.2 28.8M 85.8M
/nix/store/mjndrx4agd3l69z4l0b98vv18dfsn9kb-ffmpeg-full-6.1-data 264.9K 264.9K
/nix/store/ndwfmpylljb6hjcpy6iz4557akc9ffmp-ffmpeg-full-6.1-lib 27.7M 1.3G
/nix/store/9ryljr6yh38sssfg4ajy3pv1c089xwzf-ffmpeg-full-6.1-bin 727.2K 1.3G
# withVpl
/nix/store/d8w5qfswmgxcjqwnmqw2v9r8amrvdlpb-xgcc-13.2.0-libgcc 155.8K 155.8K
/nix/store/vqvbn2z8wyrjwvayjb2vy5krhh1kis9b-libunistring-1.1 1.7M 1.7M
/nix/store/krqp9wj3rgalmqv04y0sqw987mxsnddn-libidn2-2.3.7 352.7K 2.1M
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44 28.9M 31.1M
/nix/store/4vzal97iq3dmrgycj8r0gflrh51p8w1s-bash-5.2p26 1.5M 32.6M
/nix/store/alxp4kb0dgfczqwd7xfrpflhq50w0wck-gcc-13.2.0-libgcc 155.8K 155.8K
/nix/store/pp0jsd045xvfsz60kpbkfxbs9pbpk8z5-gcc-13.2.0-lib 8.7M 39.9M
/nix/store/b7gaf7alc4qjymv1mz79cshsayf1w20s-libvpl-2.10.1 1.1M 42.5M
/nix/store/vcyijj6xrw3acjphyb0sji1klz4xyslf-ffmpeg-full-6.1-data 264.9K 264.9K
/nix/store/smhwvfrq7j5khsk8761qiyryn5lvp7qw-ffmpeg-full-6.1-lib 27.8M 1.3G
/nix/store/rhmmr863zwlrxf8pvgfixml63zl87iwd-ffmpeg-full-6.1-bin 727.2K 1.3G
and when I change to withVpl
for my full system closure it's a 26 M improvement:
<<< /run/current-system
>>> /nix/store/r0frim4jr544rkh0fxyk7rsbd1hmqr5z-nixos-system-coffee-23.11.20240308.2be119a
Version changes:
[C.] #1 bash 5.2-p15 x2, 5.2-p21 x2, 5.2p26 -> 5.2-p15 x2, 5.2-p21 x2, 5.2p26 x2
Added packages:
[A.] #1 libvpl 2.10.1
Removed packages:
[R.] #1 intel-media-sdk 23.2.2
Closure size: 2763 -> 2764 (24 paths added, 23 paths removed, delta +1, disk usage -26.2MiB).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat! Once GPUs only supported by MFX are sufficiently legacy (or if ffmpeg were to support both), I see nothing blocking its enablement then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for all your help getting this PR merged! 😄
In order to have the most optimal setup for hardware acceleration with a recent Intel CPU then, is |
You actually just want hardware.opengl = {
enable = true;
extraPackages = with pkgs; [
onevpl-intel-gpu
];
}; and you want to override your ffmpeg to use vpl: ffmpeg-full = prev.ffmpeg-full.override {
withVpl = true; withMfx = false;
}; and that should be it, |
Ah OK, should this work for any programs that are dependent on intel media driver or media sdk as well? I did see that there was mention of it having backwards compat so it would. |
This probably depends on the use case, but for transcoding in Jellyfin,
Additionally, since I wanted to enable regular tone mapping in Jellyfin (the VPP tone mapping was too dark for me), I had to add both
This is the config I use while this PR is still underway. Mostly copied from here. boot.kernelParams = [ "i915.enable_guc=3" ];
nixpkgs.config.packageOverrides = prev: {
ffmpeg_6 = prev.ffmpeg_6.overrideAttrs (old: rec {
configureFlags =
# Remove deprecated Intel Media SDK support
(builtins.filter (e: e != "--enable-libmfx") old.configureFlags)
# Add Intel VPL support
++ [ "--enable-libvpl" ];
buildInputs = old.buildInputs ++ [
# VPL dispatcher
pkgs-unstable.libvpl
];
});
};
hardware.opengl = {
enable = true;
extraPackages = with pkgs; [
(pkgs.callPackage ../../pkgs/onevpl-intel-gpu.nix { })
intel-media-driver
intel-compute-runtime
intel-ocl
];
}; |
@programotuojes jellyfin-ffmpeg is separate from ffmpeg. It's a variant of ffmpeg_6-full. If you need VPL in jellyfin, you need to override |
In addition, I assume that it will use the same entry point as the older intel media driver and sdk, so |
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
@programotuojes: it's interesting that you say that you had to both add intel-ocl and intel-compute-runtime as opengl extra packages. I tried this as well because I'm trying to run jellyfin opencl tone mapping as well (however inside of a docker container, not as a nixos service defined in configuration.nix). Added your PR, intel-ocl and intel-compute-runtime but jellyfin tone mapping still failing for me with the same error. Additionally, if I do clinfo on the nixos machine, it seems to detect openCL for the CPU, but not on the GPU (instead of f.e. on my N5105 machine where it detects 2 opencl platforms, 1 CPU and GPU). When you do clinfo, your machine with the PR applied shows 2 opencl platforms? N100 clinfo output:
|
Nevermind, found the problem. Apparently there is an issue with intel-compute-runtime (which provides opencl libraries for intel GPU) on the latest 6.8 linux kernel: intel/compute-runtime#710 I was running that kernel and that’s why the GPU did not show up under clinfo and opencl didn’t work in the jellyfin container. Temporarily downgrading to the 6.7 kernel fixed the issue. |
Upgrading to 24.05 has caused hardware transcoding to break for me.
Has anyone else experienced this? |
@RyanGibb please create a new issue with config details |
Ah, looks like this commit changed
Changing overlay to:
Fixes this. But with #315425 we can remove the overlay and use VA-API as a hardware transcoding method (e.g. in jellfin's settings), which will use oneVPL under the hood. |
Description of changes
fresh PR continuing from #242359, sorry for the mass-ping over there :(
ping for the people who seem interested: @Atemu @PJungkamp @J-Swift
outstanding question for @midchildan: should intel-media-sdk be marked deprecated? The upstream repo is archived and suggests transitioning to onevpl
also todo: Atemu says ofborg build failed for onevpl-intel-gpu on aarch-linux, but I can't find the CI run for it. marking as draft until that's re-run and I can debug it
Things done
nix.conf
? (See Nix manual)sandbox = relaxed
sandbox = true
nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
. Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/
)