Can't capture usind VKD3D (dx12)

[ogdenwebb]

Tested with Deep Rock Galactic and Metro Exodus (Windows version) with Proton-6.9-GE-2 and Proton-6.3-4

[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Installing breakpad exception handler for appid(gameoverlayui)/version(20210519015400)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0606/212425.215633:INFO:crash_reporting.cc(270)] Crash reporting enabled for process: renderer
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0606/212425.399075:ERROR:frame_sink_video_capturer_impl.cc(206)] Invalid resolutions constraints: 0x0 must not be greater than 0x0; and also within media::limits.
ERROR: ld.so: object '/home/ogden/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
>>> Adding process 19231 for game ID 548430
[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
[obs-vkcapture] GetMemoryFdKHR failed -2
[obs-vkcapture] ------------------- vulkan capture freed -------------------
[obs-vkcapture] vk_shtex_init failed
RecordSteamInterfaceCreation (PID 19056): SteamNetworkingUtils003 / 
RecordSteamInterfaceCreation (PID 19056): SteamNetworkingUtils003 / 

[nowrep]

Can you please post log again with current master?

[ogdenwebb]

For sure :)

[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Installing breakpad exception handler for appid(gameoverlayui)/version(20210519015400)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0607/140426.005618:INFO:crash_reporting.cc(270)] Crash reporting enabled for process: renderer
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0607/140426.121604:ERROR:frame_sink_video_capturer_impl.cc(206)] Invalid resolutions constraints: 0x0 must not be greater than 0x0; and also within media::limits.
ERROR: ld.so: object '/home/ogden/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
>>> Adding process 14565 for game ID 548430
[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
RecordSteamInterfaceCreation (PID 14394): SteamNetworkingUtils003 / 
RecordSteamInterfaceCreation (PID 14394): SteamNetworkingUtils003 / 
[obs-vkcapture] Texture VK_FORMAT_B8G8R8A8_UNORM 1920x1080
[obs-vkcapture] GetMemoryFdKHR failed VK_ERROR_OUT_OF_DEVICE_MEMORY
[obs-vkcapture] ------------------- vulkan capture freed -------------------
[obs-vkcapture] vk_shtex_init failed

[ogdenwebb]

Well, now I see only black screen in my OBS.

[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
[obs-vkcapture] Injecting VK_EXT_image_drm_format_modifier extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
[obs-vkcapture] DRM format modifiers support not available
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
[obs-vkcapture] Injecting VK_EXT_image_drm_format_modifier extension
[obs-vkcapture] DRM format modifiers support not available
Installing breakpad exception handler for appid(gameoverlayui)/version(20210608222338)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0614/214140.839177:INFO:crash_reporting.cc(270)] Crash reporting enabled for process: renderer
Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
[0614/214140.961349:ERROR:frame_sink_video_capturer_impl.cc(206)] Invalid resolutions constraints: 0x0 must not be greater than 0x0; and also within media::limits.
ERROR: ld.so: object '/home/ogden/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
>>> Adding process 1457 for game ID 548430
[obs-vkcapture] Init 0.6
[obs-vkcapture] Injecting VK_KHR_external_memory_fd extension
[obs-vkcapture] Injecting VK_EXT_image_drm_format_modifier extension
Fossilize INFO: Overriding serialization path: "/mnt/evo/SteamLibrary/steamapps/shadercache/548430/fozpipelinesv5/steamapprun_pipeline_cache".
[obs-vkcapture] DRM format modifiers support not available
RecordSteamInterfaceCreation (PID 1273): SteamNetworkingUtils003 / 
RecordSteamInterfaceCreation (PID 1273): SteamNetworkingUtils003 / 

[nowrep]

I've changed socket path (it's in $XDG_RUNTIME_DIR now), make sure you are also using updated OBS plugin.

[ogdenwebb]

I've changed socket path (it's in $XDG_RUNTIME_DIR now)

Do I need to do something on my side considering these changes?

make sure you are also using updated OBS plugin.

I build package from AUR, so obs-vkcapture-git-r77.9fdfef8-1is installed on my system, and as I can see, it's the most recent version from git.

[nowrep]

Alright, I'll see what's up with Steam.

For now I guess ln -s $XDG_RUNTIME_DIR/obs-vkcapture.socket /tmp after launching OBS should fix it.

[nowrep]

Or revert 168baa1

[ogdenwebb]

Well, I've done a few quick tests: with vkcube and native Valheim it works, but now it doesn't work with any Proton game (I tested with Deep Rock Galactic, Gunfire Reborn and Grim Dawn at least).

[ogdenwebb]

Now it works. :)