Skip to content

Standalone NVENC encoder (0.21.0+) broke NVENC on Hyper-V with GPU passthrough #2141

Description

@devusr1x

Is there an existing issue for this?

  • I have searched the existing issues

Is your issue described in the documentation?

  • I have read the documentation

Is your issue present in the nightly release?

  • This issue is present in the nightly release

Describe the Bug

NVENC is not being properly detected on a Hyper-V Windows 11 machine with GPU passthrough:

  • 0.20.0 works fine and I can see the GPU encoder being used, same thing for Parsec. HEVC works fine when connecting with Moonlight.
  • 0.21.0 and the latest nightly version that I tested (69a3edd) completely ignore NVENC even if you try to force it. I have played around with dummy monitors, HAGS (on/off), different driver versions and nvidia-patch to see if I can trick Sunshine into using NVENC, but nothing works. Again, both Parsec and 0.20 work properly, so the issue must be related with the new NVENC encoder (Add standalone NVENC encoder #1427). When connecting with Moonlight, it reverts to H264 with Software encoding without giving me any warnings, resulting in high CPU usage.

Expected Behavior

NVENC is properly detected and used.

Additional Context

No response

Host Operating System

Windows

Operating System Version

11 23H2

Architecture

64 bit

Sunshine commit or version

0.20.0 and #69a3edd

Package

Windows - installer

GPU Type

Nvidia

GPU Model

GTX 1060 6GB

GPU Driver/Mesa Version

516.93 (Studio)

Capture Method (Linux Only)

No response

Config

nv_preset = p7
origin_web_ui_allowed = pc
nv_tune = hq
adapter_name = NVIDIA GeForce GTX 1060 6GB
min_threads = 4
av1_mode = 0
fec_percentage = 20
hevc_mode = 0
encoder = nvenc
install_steam_audio_drivers = disabled

Apps

No response

Relevant log output

This log is from nightly (#69a3edd):

[2024:02:14:00:35:46]: Info: nvprefs: Opened undo file from previous improper termination
[2024:02:14:00:35:46]: Info: nvprefs: OGL_CPL_PREFER_DXPRESENT has been changed from our value in base profile, not restoring
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:14:00:35:46]: Error: nvprefs: Failed to restore global profile settings from undo file, deleting the file anyway
[2024:02:14:00:35:46]: Info: nvprefs: Changed PREFERRED_PSTATE to PREFERRED_PSTATE_PREFER_MAX for sunshine.exe
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:14:00:35:46]: Error: nvprefs: Couldn't save application profile settings
[2024:02:14:00:35:46]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:14:00:35:46]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:14:00:35:46]: Error: nvprefs: Couldn't save global profile settings
[2024:02:14:00:35:46]: Info: Compiling shaders...
[2024:02:14:00:35:46]: Info: System tray created
[2024:02:14:00:35:47]: Info: Compiled shaders
[2024:02:14:00:35:47]: Info: Trying encoder [nvenc]
[2024:02:14:00:35:47]: Info: ddprobe.exe [1] [] returned: 0x00000000
[2024:02:14:00:35:47]: Info: Set GPU preference: 1
[2024:02:14:00:35:47]: Info: 
Device Description : NVIDIA GeForce GTX 1060 6GB
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001C03
Device Video Mem   : 6052 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16341 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:14:00:35:47]: Info: Active GPU has HAGS enabled
[2024:02:14:00:35:47]: Info: Using realtime GPU priority
[2024:02:14:00:35:47]: Info: Desktop resolution [1920x1080]
[2024:02:14:00:35:47]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:14:00:35:47]: Info: Display refresh rate [60Hz]
[2024:02:14:00:35:47]: Info: Requested frame rate [60fps]
[2024:02:14:00:35:47]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.149414,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:14:00:35:47]: Info: SDR color coding [Rec. 601]
[2024:02:14:00:35:47]: Info: Color depth: 8-bit
[2024:02:14:00:35:47]: Info: Color range: [JPEG]
[2024:02:14:00:35:47]: Error: NvEncOpenEncodeSessionEx failed
[2024:02:14:00:35:47]: Info: SDR color coding [Rec. 601]
[2024:02:14:00:35:47]: Info: Color depth: 8-bit
[2024:02:14:00:35:47]: Info: Color range: [JPEG]
[2024:02:14:00:35:47]: Error: NvEncOpenEncodeSessionEx failed
[2024:02:14:00:35:47]: Info: Encoder [nvenc] failed
[2024:02:14:00:35:47]: Error: Couldn't find any working encoder matching [nvenc]
[2024:02:14:00:35:47]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:02:14:00:35:47]: Info: Trying encoder [quicksync]
[2024:02:14:00:35:47]: Info: 
Device Description : NVIDIA GeForce GTX 1060 6GB
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001C03
Device Video Mem   : 6052 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16341 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:14:00:35:47]: Info: Active GPU has HAGS enabled
[2024:02:14:00:35:47]: Info: Using realtime GPU priority
[2024:02:14:00:35:47]: Info: Desktop resolution [1920x1080]
[2024:02:14:00:35:47]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:14:00:35:47]: Info: Display refresh rate [60Hz]
[2024:02:14:00:35:47]: Info: Requested frame rate [60fps]
[2024:02:14:00:35:47]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.149414,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:14:00:35:47]: Info: Encoder [quicksync] is not supported on this GPU
[2024:02:14:00:35:47]: Info: Trying encoder [amdvce]
[2024:02:14:00:35:47]: Info: 
Device Description : NVIDIA GeForce GTX 1060 6GB
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001C03
Device Video Mem   : 6052 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16341 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:14:00:35:47]: Info: Active GPU has HAGS enabled
[2024:02:14:00:35:47]: Info: Using realtime GPU priority
[2024:02:14:00:35:47]: Info: Desktop resolution [1920x1080]
[2024:02:14:00:35:47]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:14:00:35:47]: Info: Display refresh rate [60Hz]
[2024:02:14:00:35:47]: Info: Requested frame rate [60fps]
[2024:02:14:00:35:47]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.149414,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:14:00:35:47]: Info: Encoder [amdvce] is not supported on this GPU
[2024:02:14:00:35:47]: Info: Trying encoder [software]
[2024:02:14:00:35:48]: Info: 
Device Description : NVIDIA GeForce GTX 1060 6GB
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001C03
Device Video Mem   : 6052 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16341 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:14:00:35:48]: Info: Active GPU has HAGS enabled
[2024:02:14:00:35:48]: Info: Using realtime GPU priority
[2024:02:14:00:35:48]: Info: Desktop resolution [1920x1080]
[2024:02:14:00:35:48]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:14:00:35:48]: Info: Display refresh rate [60Hz]
[2024:02:14:00:35:48]: Info: Requested frame rate [60fps]
[2024:02:14:00:35:48]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.149414,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:14:00:35:48]: Info: SDR color coding [Rec. 601]
[2024:02:14:00:35:48]: Info: Color depth: 8-bit
[2024:02:14:00:35:48]: Info: Color range: [JPEG]
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] profile High, level 4.2, 4:2:0, 8-bit
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] frame I:1     Avg QP:31.00  size:  1278
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] mb I  I16..4: 99.9%  0.1%  0.0%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] 8x8 transform intra:0.1%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] i16 v,h,dc,p: 94%  0%  6%  0%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  0%  0% 75% 12%  0%  0%  0%  0% 12%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  0%  0% 100%  0%  0%  0%  0%  0%  0%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] i8c dc,h,v,p: 100%  0%  0%  0%
[2024:02:14:00:35:48]: Info: [libx264 @ 00000217a8f24e40] kb/s:613.44
[2024:02:14:00:35:48]: Info: 
Device Description : NVIDIA GeForce GTX 1060 6GB
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001C03
Device Video Mem   : 6052 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16341 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:14:00:35:48]: Info: Active GPU has HAGS enabled
[2024:02:14:00:35:48]: Info: Using realtime GPU priority
[2024:02:14:00:35:48]: Info: Desktop resolution [1920x1080]
[2024:02:14:00:35:48]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:14:00:35:48]: Info: Display refresh rate [60Hz]
[2024:02:14:00:35:48]: Info: Requested frame rate [60fps]
[2024:02:14:00:35:48]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.149414,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:14:00:35:48]: Info: 
[2024:02:14:00:35:48]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:02:14:00:35:48]: Info: 
[2024:02:14:00:35:48]: Info: Found H.264 encoder: libx264 [software]
[2024:02:14:00:35:48]: Info: Configuration UI available at [https://localhost:47990]
[2024:02:14:00:35:49]: Info: Registered Sunshine mDNS service

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions