Intel GPU Top: No activity for VideoEnhance even with VA-API working

[jike212]

System information

• CPU information: 11th Gen Intel(R) Core(TM) i7-1185G7
• GPU information: Intel Corporation TigerLake-LP GT2 [Iris Xe Graphics]
• Display server if rendering to display: Display: wayland server: X.Org v: 1.21.1.4 with: Xwayland v: 21.1.4
• OS: Fedora 35

Issue behavior

I have VAAPI working and I can see video decoding activity in the Video bar in intel_gpu_top indicating that the GPU is properly decoding the file (occurs both with MPV and VLC). There is no activity in VideoEnhance section however. It is as though that part of the GPU is not being activated and it's not clear why.

Describe the expected behavior

Both Video and VideoEnhance should show activity when a file with a supported codec is being played. When I was running Arch with X11, I felt like VideoEnhance also showed activity so I'm unsure if the issue is specific to Wayland.

Debug information

• What's libva/libva-utils/gmmlib/media-driver version?: 2.13.0-3.fc35
• Could you confirm whether GPU hardware exist or not by ls /dev/dri?
• Could you provide vainfo log if possible by vainfo >vainfo.log 2>&1?: Yes, the following appears after running that command by-path card0 renderD128

[ghost]

on a intel i7-1165g7, same OS, I don't see a "VideoDecode" field on intel_gpu_top, there's VideoEnhance though.

[jike212]

@m911t: Yes, you are correct, it's VideoEnhance. I've updated the original issue.

[XinfengZhang]

video enhance is vebox which is used to handle vpp workload, including de-interlacing, de-noise etc.
it also could work with SFC to provide scaling, CSC , zoom support.
whether it is busy depend on the task you are running, from the description, it just run video playback, so... it is expectation .

[jike212]

Interesting, even when I run the following, there's still no activity on VideoEnhance:

mpv --hwdec=auto profile=gpu-hq scale=ewa_lanczossharp cscale=ewa_lanczossharp video-sync=display-resample interpolation tscale=oversample <file name>

I wonder if it's an issue with FFMPEG then? Does it require building FFMPEG with oneVPL?

[ghost]

Indeed! Even if you enable mpv's video enhancements, VideoEnhance never gets used. All is done by the 3D Render engine... Which leads to overusage and dropped frames. The only time I've seen that engine be used was when vaapi decoding was working with Chromium browsers, but I use wayland now and only Firefox has working vaapi decoding with wayland.

[jike212]

Indeed! Even if you enable mpv's video enhancements, VideoEnhance never gets used. All is done by the 3D Render engine... Which leads to overusage and dropped frames. The only time I've seen that engine be used was when vaapi decoding was working with Chromium browsers, but I use wayland now and only Firefox has working vaapi decoding with wayland.

Are you disabling the RDD sandbox for hardware video acceleration in Firefox? As far as I'm aware, that is still a requirement for the time being. Otherwise video acceleration doesn't work in Firefox yet.

[XinfengZhang]

in media driver , there are multiple path video processing , for example,
we could choose scaling to use EU (render), vebox+SFC,
even with EU patch, there maybe different 3D sampler to support scaling, it is different with platforms, for example:
there are AVS sampler on KBL, so, EU may call AVS to have a better quality, also could use bilinear to have a simple scaling.
so , I m not surprise to see no task on VEBox.
there are some definitions in va_vpp.h named VAProcMode.

[ghost]

@jike212 yeah I am.

[ghost]

@XinfengZhang are you aware of any configuration in mpv that makes use of the videoEnhance engine?

[XinfengZhang]

TBH, I also dont know :). maybe we need to try the commands to check .

[ghost]

hmm. i've used common settings but they dont ever seem to use videoEnhance.
is there a list of things that the videoEnhance engine can do, to do some experimenting?

[XinfengZhang]

from my understanding, if there are denoise, de-interlace ... it will surely to through video enhancement engine.

[eero-t]

E.g. for FFmpeg, there's a separate "-scale_vaapi" (or "-scale_qsv") option that needs to be used to request scaling & format conversions to be done on GPU video HW. Using it requires some expertise, as I've seen example commands which actually ask FFmpeg to ping-pong the data between CPU & GPU, instead of doing everything on GPU side and getting only end result back to CPU side.

[kkanungo17]

mpv uses its own scalers and shaders to their processing. They are quite intent on not wanting to rely on built-in hardware processing as much as possible, to the point of not even enabling hardware decoding if possible.

You can ask in their repo for an option to use intel media processing, but I’m not sure if anyone would be interested to.

[lavilao]

write this on a terminal mpv -vf=help | grep vaapi, the shown filters will use videoenhance. Note that it will only work on videos that are decoded through vaapi, if you use sw decoding or vaapi-copy it will not work. An example would be mpv -no-config -hwdec=auto -vf=denoise_vaapi,scale_vaapi=1366:768:nv12:hq nameofvideo.mp4

[kkanungo17]

@lavilao Interesting.

denoise_vaapi yields activity in VideoEnhance, but scale_vaapi does not. Additionally, there doesn't seem to be any difference between normal and hq modes.

It still looks quite good, comparable to my mpv settings. Is it a hardware reason that sw decoding can't be made to work with scale_vaapi?

[ghost]

because for intel gpus, it's scale_qsv

[lavilao]

@kkanungo17 yes, I also dont notice any difference between default and hq, as for the second part the reason is the surface format, hw decoded media format is nv12 while sw decoded is yuv420p + a lot more.If you try to use any hw filter while on sw decoding mpv will throw an error telling you that could not convert between surfaces or something like that.
@m911t not on my celeron :P

[ghost]

yeah soz, it can be _vaapi and _qsv but _qsv needs a ffmpeg compiled with some extra stuff

[ghost]

ok, set "vf=scale_vaapi=1920:1080::hq::2" and it creates activity on VideoEnhance, very nice. Perhaps not the best place to ask, but does this conflict with mpv's scale=,cscale=,dscale=,tscale= options?

[kkanungo17]

@lavilao @m911t So I do have scale_qsv option available, but that doesn't work at all, always returning the "could not convert between surfaces" error. And vf=scale_vaapi=1920:1080::hq::2 doesn't create activity on VideoEnhance for me.

But looks like mpv has a vo=vaapi option, which you can pass vo-vaapi-scaling=hq as a suboption, and that seems to engage the VideoEnhance engine. Too bad that mode is virtually unusable because of no OSD or subtitle support.

but does this conflict with mpv's scale=,cscale=,dscale=,tscale= options?

If you scale beyond what you've passed to the vf option, mpv's scaling options should kick in. Use video-unscaled to prevent that.

[lavilao]

@kkanungo17 that surface error is typically shown when you are trying to use a hardware filter on a software decoder, are you sure the file you are testing is hardware decoded by your pc? Ironically vo=vaapi was usable before, but after mpv 0.32 or 0.33 it became unusable (no osd and that), I think using vo=gpu and hwdec=vaapi should be enough to use scale_vaapi, about qsv I dont know, my cpu is not compatible on linux with qsv.

[kkanungo17]

@lavilao I understand that, and scale_vaapi works fine with hwdec=vaapi for me. I was just talking about scale_qsv.

I do wonder what it is about that vo=vaapi engages VideoEnhance but vf=scale_vaapi=mode=hq does not.

[lavilao]

@kkanungo17 I think scale_qsv requires mpv to be compiled with it supported (if it even is supported). About vo=vaapi I guess you can check the vo vaapi file on the source code ¯_(ツ)_/¯

[ghost]

yeah scale_qsv requires ffmpeg built with it

[LhGu]

If you want to use qsv to scale ("Intel Quick Sync Video"), you must set up ffmpeg with QSV enablement(--enable-libmfx). For vaapi, lavilao's answer is correct. If everything is ok, you could see scaling workloads in VideoEnhance.

[LhGu]

Add ffmpeg with QSV wiki: https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[pik10]

I don't see any VideoEnhance in Firefox 110, Fedora 37, VP9 or AV1 playback on YouTube. Only Video shows some activity. I see VideoEnhance in Chromium or Chrome in VP9 playback. My CPU is Intel 11th Gen.

[dm17]

On a 12th gen intel I had to add the following to get anything to show for videoenhance during videos in chromium:

--disable-features=UseChromeOSDirectVideoDecoder
--enable-features=VaapiVideoDecodeLinuxGL
--enable-features=VaapiIgnoreDriverChecks

Still unnecessarily bad performance according to ufotest.com (obviously capped by iGPU since it goes to 99% render/3d in that case). Also YouTube videos end up clipping @ when it gets to 99% on render/3d, despite being vaapi enabled. This did not happen before the 3 above flags were added, despite gpu acceleration still being enabled in chrome.