Windows Intel QSV hardware transcode fail #2626
Comments
frankyifei
changed the title
|
Are you using a headless system? Is your graphics card driver up to date? |
No, I'm using windows 10 and all drivers up to date. |
|
Test command line in windows console, I found the current FFMpeg does not support scale_qsv run this command will give an error I then change the scale_qsv in first command into vpp_qsv and the hardware accelerated encoding works The code responsible for this behaviour is here: |
|
This is due to some problems about the qsv encoder in the ffmpeg command line, which needs improvement. I will look into it when I have time. Similar problems have been solved on VAAPI. |
@nyanmisaka The problem is that subtitles is a software filter so it is necessary to download the frame from the video card to the system, overlay the subtitles, then upload to video card again. I've been trying to get a dev's eyes on my solution so I'm hoping my reply here will do just that. |
Thanks for your suggestion. I know it can be solved in this way, and before that I want to try to find if there are other more efficient solutions. |
|
I modified MediaBrowser.Controller now that jellyfin use vpp_qsv instead of scale_qsv. This command line works if you type it in windows console but jellyfin does not work. It still says error. |
|
This is hardware decode and encode command： Error in Logs: This is software decode and hardware encode command: Error in Logs: Both works in console and not in Jellyfin. I wonder there is something missing here. The bold text is where I modify the command. Because I modified EncodingHelper.cs, I thought the same command should also work in Jellyfin. But it does not work. |
I think it is a different problem here. With the two command lines, Subtitles work perfectly but only if you type in console. Why are there different outcomes? |
|
On my HD630 iGPU notebook, these commands works fine on both console and JF. Do you have other discrete graphics cards installed on your computer? It may be that the conflict between drivers prevents ffmpeg from using iGPU. You can force |
|
I only have HD600 on my NAS. The commands works for me now. |
|
I'm so happy seeing all this work on the problem that's been plaguing me for weeks now, starting with Plex (its clients can't play videos for sh!t, if it's not audio not playing, it's transcoding crashing plex itself). |
I also found a problem related to this one. The QSV transcode cannot work from JellyfinTray. With the above fix, it still not transcoding and gives a error of "failed to create Direct3d device". Then I found a post about this from FFmpeg https://trac.ffmpeg.org/ticket/6827 |
If you choose to newly install Jellyfin as a system network service, does ffmpeg qsv work properly? |
I can confirm that hardware transcoding is not possible when running as a service. I found the same problem when I used NSSM on Plex. The compromise that I ended up with was to use RBTray to send Jellyfin to the system tray |
It does happen, but he cannot use ffmpeg qsv on his computer even with the tray. Have you encountered similar issues?
|
|
Sorry, maybe a false alarm about the Jellyfin tray. It was the command that use QSV for both decoding and encoding. Specifically: What adding the bold parts do is after scaling the frame via QSV, we download the frame from Video RAM to System RAM, do a software overlay of the subtitles, then upload it back up to Video Ram for output processing. |
I found a way to use the QSV decoder and encoder and improve the text subtitle burning performance. Unlike using You can try the following commands to see the transcoding speed and hardware utilization. no downscaling downscaling to 720p This method uses QSV as a hw decoder and hw encoder, and will also take up 40% ~ 55% of my i5 7300hq to render text subtitles. When the original video is 1080p, the transcoding rate can reach 100fps ~ 180fps depending on the original video encoding format and resolution. Compared to using |
|
My Nas was not as powerful so it only runs at about speed=2x |
For a NAS with a low power processor, this performance is already very good when burning subtitles. Does the MFX error still exist in Jellyfin? |
it is still there. it only happens when I set hw decoding + hw encoding. Now I use soft decoding + hw encoding. Because the current command jellyfin will cause a error of I suspect it is the reason that cause MFX error |
This is a filter related error I known. But MFX error is unable to create a QSV device. Here is the reason
I just tried to install Jellyfin as a service and got similar MFX errors. |
|
Here's the quicklink for the ticket. What do you think @anthonylavado |
|
@Artiume I’ve never tried QSV myself, so this is news. The Microsoft link you’ve posred says this should work starting with DX11.1, but who knows if that’s getting called by FFMpeg or not. I guess not. I have a couple of ideas to try and fix this from the Windows service side, but they require some build changes. |
I think the note about DX11.1 is that WARP is supported in DX11.1, but that is a software rasterizer...
I have an i7-7500U in a NUC form factor and I tried the two commands @nyanmisaka suggested. The no scaling: renders at 38-43 FPS with 19-24% CPU utilization Does "scale=1280:720" mean it is doing the scaling in software? Edit: To append to this, I tried the scaling command on a 1080p source with an external SRT subtitle and the results were: renders at 40-43 fps but the CPU goes to 27% Edit 2: One more edit. I tried the command I originally posted that used hwdownload/hwupload on the 1080p source with srt subtitle and the results were: |
The conversion speed of U-series low-voltage processors is usually lower than that of HQ-series processors. If it is a desktop processor, it will perform even better. Memory frequency and channel number may also be influencing factors. I use the command you suggest to get only 40fps under the same conditions, but the command I recommend can reach at least 100fps. Using |
That's interesting that it is a higher cost to hwdown/hwupload than software scaling. Thanks for all this info. |
@frankyifei @Sp4rky001 https://ffmpeg.zeranoe.com/builds/win64/shared/ffmpeg-latest-win64-shared.zip |
|
Ok, it should be someting upstream |
|
I just wanted to share a few things that I learned in the last few days. As I've been playing various files, including x265, and Hi10 with image-based subtitles that require overlay, I've been modifying the source code and building Jellyfin to fit my own needs. A few conclusions that I've arrived at:
With these changes, a text-based subtitle video transcodes at 140fps at around 48% cpu while an image-based subtitle video transcodes at 90fps at around 40% cpu, all while running Jellyfin as a service created manually using NSSM. I think this is acceptable performance and still preferable over software transcoding, which takes 100% cpu. These are just my suggested changes and I'm attaching my patch file for reference. |
I ran into the same problem with image-based subtitles and in my own code, I implemented the changes that you describe here (remove hwupload and change overlay_qsv to overlay). While this works beautifully, I get this annoying warning with EVERY SINGLE FRAME that's transcoded:
any idea how to avoid this warning? I've tried all sorts of combinations of parameters but still get this warning. I can try to add -loglevel warning but I'm still looking for the best place in the code to add it. |
The problem we discussed earlier is the use of example: Linux + ffmpeg 4.3 is requied for the above commands and the reasons are explained earlier. |
|
@nyanmisaka , since the problem of overlay_qsv is from upstream ffmpeg, can we avoid using it on windows machine to work around this problem for now until it gets fixed by ffmpeg developers. I know there is an increase of CPU load but it is better to have it work for now. at least it still use gpu for decoding and encoding. |
|
This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments. |
|
I have to revive this. Server is running on windows 10, client is Chrome. The movie contains vodsub subtitles. Now the error message is And the command in use is this
|
|
are you trying 10.6.4 or 10.7.0-rc1 |
10.6.4, I think there is not any new change in 10.7.0 |
|
on 10.7 , the error message is |
|
I cannot reproduce that on Intel 7th Gen HD 630. Update your driver an try again. |
|
Hi again! In the past few months, I've settled on a setup that I can live with. My setup is a Windows 10 on a 6th gen i7 with embedded Intel graphics. I run in headless mode and access the PC through remote desktop. What I discovered is that graphics acceleration only really works when you have an active desktop session that's logged in. The video driver just isn't fully accessible in a locked session or in a remote desktop session. I verified this with a video that showed blue when I logged in through remote desktop but transcodes just fine when logged in to the pc. So because I remote into the PC, Jellyfin ends up running in a remote desktop session. My solution was to use RDP Wrap which tricks Windows into allowing multiple interactive sessions. I created a user named Jellyfin that logs in to the PC automatically on bootup (control userpassword2) and have Jellyfin start automatically, then set the computer to never sleep or log out. I'm then free to log in through remote desktop with a second account for everything else I need to do. With this, I've had almost no issues transcoding the various videos that I threw at it. I hope this helps. |
|
Good idea! I have looked into the patchwork of ffmpeg. Intel stuffs have sent a series of patches to make it possible to init QSV on D3D11VA, which supports headless mode well. But it's still pending due to some discussions. |
I too normally remote desktop (RD) into my server from my laptop, though my server has a display connected. I was reading this thread via RD into my server as I was having some issues with QSV and ffmpeg (looks to be a subtitle issue). I played a video on a client while logged in with RD and it did transcode pretty normally. I then logged out of RD and tried playing the same file and it errored out. I then logged into an active session on my server and played the same file on the same client and it worked just fine. I bet we can't be the only ones experiencing this issue. Is this in any of the guides or install instructions? I'll give your solution a try since I remote into the server for purposes other than just Jellyfin. What is "RDP Wrap"? |
|
@wlau2929 Hey! There is a fix for QSV on headless machine. Would you like to try it? Backup your data first. Download the jellyfin-ffmpeg 4.3.1-4 from the link below and extract all into your jellyfin installation folder. Restart jellyfin and give a try on QSV. |
This works for me in most cases except dvdsub subtitles transcoding stills fails, on Intel j4105. |
|
@frankyifei Any logs? |
using hardware decoding and encoding at the same time does not work. using cpu decoding and qsv encoding works. I don't know if this is caused by my machine's low spec or something else. |
|
Does the error only happen to this video? do {
err = MFXVideoCORE_SyncOperation(s->session_download, sync, 1000);
} while (err == MFX_WRN_IN_EXECUTION);
if (err < 0) {
av_log(ctx, AV_LOG_ERROR, "Error synchronizing the operation: %d\n", err);
return AVERROR_UNKNOWN;
} |
|
@nyanmisaka, I tried two other movies and both had same error. This is one of the log |
I've looked at your nyanmisaka/jellyfin-ffmpeg@98afe79 commit but I couldn't understand why a "custom hwupload filter" would fix this, then I noticed you bumped the intel dependencies versions ... after some digging I found Intel-Media-SDK/MediaSDK@e3f10e7 commit. I wanted to ask you if this is the change that now allows for QSV to work headless ? just wanting to better understand |
|
Hi! The jellyfin-ffmpeg repo is only for Linux for the time being. So I didn't submit the Windows specific patches there. Please reference to this branch. I have merged some DX11 QSV stuff there. |
|
@Pljushevij open your own thread/issue man. |
Sorry, man! :) |
…icitly select d3d11va/DX11 device type
UPD: Rebase of last patch set over current master and use DX9 as default device type.
Makes selection of dxva2/DX9 device type by default as before with explicit d3d11va/DX11 usage to cover more HW configurations.
Added warning message to expect changing default device type in the future.
Fixes TGL / AV1 decode as requires DX11 with explicit DX11 type
selection.
Add headless/multi adapter support and fixes:
https://trac.ffmpeg.org/ticket/7511
https://trac.ffmpeg.org/ticket/6827
http://ffmpeg.org/pipermail/ffmpeg-trac/2017-November/041901.html
https://trac.ffmpeg.org/ticket/7933
InitialForce@338fbcd
jellyfin/jellyfin#2626 (comment)
Any other fixes are welcome including OpenCL interop patch since I don't have proper setup to validate this use case
Decoding, encoding, transcoding have been validated.
child_device_type option is responsible for d3d11va/dxva2 device selection
Usage examples:
DirectX 11:
-init_hw_device qsv:hw,child_device_type=d3d11va
-init_hw_device qsv:hw,child_device_type=d3d11va,child_device=0
OR
-init_hw_device d3d11va=dx -init_hw_device qsv@dx
DirectX 9 is still supported but requires explicit selection:
-init_hw_device qsv:hw,child_device_type=dxva2
OR
-init_hw_device dxva2=dx -init_hw_device qsv@dx
Signed-off-by: Artem Galin <artem.galin@intel.com>
Describe the bug
The transcode failed with the error in log.
[AVHWDeviceContext @ 000001a1363ace00] Error initializing an MFX session: -3. Error creating a QSV deviceIt is strange that when I ran the same command in console there is no such error.
System (please complete the following information):
To Reproduce
anything requires transcode, if I disable hardware transcode, it works
Logs
Another logs shows:
Seems similar to this problem
https://trac.ffmpeg.org/ticket/8379
The text was updated successfully, but these errors were encountered: