Gamescope crashes in fullscreen (NVIDIA)

[LethalManBoob]

@archlinux ~ % gamescope -h 1080 -H 1080 -f -- vkquake
No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
wlserver: [backend/headless/backend.c:82] Creating headless backend
vulkan: selecting physical device 'NVIDIA GeForce RTX 3070': queue family 2
vulkan: physical device supports DRM format modifiers
vulkan: vkGetPhysicalDeviceFormatProperties2 returned zero modifiers for DRM format 0x3231564E (VkResult: 0)
vulkan: supported DRM formats for sampling usage:
vulkan: 0x34325241
vulkan: 0x34325258
wlserver: [wayland] unable to lock lockfile /run/user/1000/gamescope-0.lock, maybe another compositor is running
wlserver: Running compositor on wayland display 'gamescope-1'
wlserver: [backend/headless/backend.c:18] Starting headless backend
wlserver: [xwayland/sockets.c:63] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use
wlserver: [xwayland/sockets.c:63] Failed to bind socket @/tmp/.X11-unix/X1: Address already in use
wlserver: [xwayland/server.c:92] Starting Xwayland on :2
wlserver: [types/wlr_surface.c:741] New wlr_surface 0x560905bf3b90 (res 0x560905bea290)
wlserver: [xwayland/server.c:250] Xserver is ready
pipewire: stream state changed: connecting
pipewire: stream state changed: paused
pipewire: stream available on node ID: 144
Command line: vkquake
Found SDL version 2.24.1
Detected 8 CPUs.
Initializing vkQuake v1.20.3
Built with GCC 12.2.0
Host_Init
Playing registered version.
Console initialized.
pipewire: renegotiating stream params (size: 1920x1080)
UDP4 Initialized
UDPv6 Initialized
Server using protocol 999+ (FTE-RMQ)
Exe: 11:40:25 Oct 14 2022
wlserver: [types/wlr_surface.c:741] New wlr_surface 0x560905aa23e0 (res 0x560905bec200)

Vulkan Initialization
Using Vulkan 1.1
Instance extensions:
VK_KHR_surface
VK_KHR_xlib_surface
VK_KHR_get_surface_capabilities2
VK_KHR_get_physical_device_properties2

Vendor: NVIDIA
Device: NVIDIA GeForce RTX 3070
Driver: NVIDIA 520.56.06
Using subgroup operations
Device extensions:
VK_KHR_swapchain
VK_KHR_get_memory_requirements2
VK_KHR_dedicated_allocation
VK_EXT_subgroup_size_control
Using A2B10G10R10 color buffer format
Using D32_S8 depth buffer format

Creating command buffers
Initializing staging
Creating descriptor set layouts
Reallocating dynamic VBs (256 KB)
Reallocating dynamic IBs (1024 KB)
Reallocating dynamic UBs (256 KB)
Initializing samplers
Texture lod bias: 0.000000
Creating pipeline layouts
Allocating lightstyles buffer (0 KB)
Allocating lights buffer (4 KB)

Sound Initialization
Using IMMEDIATE present mode
Creating color buffer
AA disabled
Creating depth buffer
Creating render passes
Creating frame buffers
Creating pipelines
SDL audio spec : 44100 Hz, 1024 samples, 2 channels
SDL audio driver: pulseaudio - Starship/Matisse HD Audio Controller Digital Stereo (IEC958), 65536 bytes buffer
Audio: 16 bit, stereo, 44100 Hz
CDAudio disabled at compile time

Language initialization
Couldn't load 'localization/loc_english.txt'
from '/home/joseph'

========= Quake Initialized =========

execing quake.rc
execing default.cfg
The XKEYBOARD keymap compiler (xkbcomp) reports:

Warning: Unsupported maximum keycode 708, clipping.
X11 cannot support keycodes above 255.
Errors from xkbcomp are not fatal to the X server

[Rodancoci]

Experiencing the same issue:

$ gamescope -H 1440 -f -- glxgears
No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
wlserver: [backend/headless/backend.c:82] Creating headless backend
vulkan: selecting physical device 'NVIDIA GeForce RTX 3060 Ti': queue family 2
vulkan: physical device supports DRM format modifiers
vulkan: vkGetPhysicalDeviceFormatProperties2 returned zero modifiers for DRM format 0x3231564E (VkResult: 0)
vulkan: supported DRM formats for sampling usage:
vulkan:   0x34325241
vulkan:   0x34325258
wlserver: Running compositor on wayland display 'gamescope-0'
wlserver: [backend/headless/backend.c:18] Starting headless backend
wlserver: [xwayland/sockets.c:63] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use
wlserver: [xwayland/server.c:92] Starting Xwayland on :1
wlserver: [types/wlr_surface.c:748] New wlr_surface 0x5570788c8780 (res 0x5570788be640)
wlserver: [xwayland/server.c:250] Xserver is ready
pipewire: stream state changed: connecting
pipewire: stream state changed: paused
pipewire: stream available on node ID: 137
pipewire: renegotiating stream params (size: 2560x1440)
wlserver: [types/wlr_surface.c:748] New wlr_surface 0x55707877b6b0 (res 0x5570788c17e0)
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning:          Unsupported maximum keycode 708, clipping.
>                   X11 cannot support keycodes above 255.
Errors from xkbcomp are not fatal to the X server

Running in borderless mode with -b will work but toggling fullscreen mode with Super+F will cause execution to freeze.

[LethalManBoob]

Im surprised such a big issue is not getting fixed yet they bothered to implement nvidia NiS.

[lostalejandro]

We need more details to be able to help you guys. Which distro/kernel version are you running? also which driver version? seems like you both have RTX 3000 series so i guess it's something about the drivers not gamescope

[LethalManBoob]

We need more details to be able to help you guys. Which distro/kernel version are you running? also which driver version? seems like you both have RTX 3000 series so i guess it's something about the drivers not gamescope

Somone said its an nvidia driver issue causing gamescope to have issues. But yes i am running a 3070. i currently run arch with the 6.0.5-1-cachyos-pds-lto kernel but different kernels have not changed the nature of the fullscreen crashing ive experienced.
nvidia driver 520.56.06

[Rodancoci]

Sorry yes, the relevant info is:

• OS: Arch Linux
• Kernel: 6.0.5-arch1-1
• Nvidia driver 520.56.06
• DE: GNOME 42.5
• GPU: Nvidia 3060 Ti

[Bitwolfies]

I cant get ANY part of gamescope to work on this new NV driver, so it runs deeper than fullscreen.

[lostalejandro]

I cant get ANY part of gamescope to work on this new NV driver, so it runs deeper than fullscreen.

Well as seem in the above comments, it might be something on rtx 3000 series and latest Nvidia driver because I'm running gamescope on my gtx 1660 super without any issues, it even runs in embedded mode (which it was impossible a few months ago, still has flickering but it works!). Nvidia driver 515.76 was giving me a lot of issues, so I was happy when I updated to 520.56. You guys could try Nvidia-all script from frogging family and try an stable version like 515.49 or something just to see if it's a driver issue.

[jfaz1]

Also have this issue on Fedora 37 with 520 (3070 ti). CS:GO doesn't launch at all, whether in fullscreen or not. However other games (e.g. Castle Crashers) work fine in borderless but NOT in fullscreen.

[famellad]

Still a problem with the 525 beta drivers and a 3060

[the-cave-dweller]

Back when I used an NVIDIA card I was able to play on gamescope by installing the egl variant of wlroots, using the normal one resulted in constant crashes, in case it helps anyone.

[MCPO-Spartan-117]

Nvidia has issues with resizing some applications like wine and gamescope, often crashing them and giving X11 BadAccess errors, might be hitting into a similar issue.

[tsawhill]

I am seeing this issue with NVIDIA driver 525.60.11, an RTX 3080, and EndeavourOS with the latest gamescope built from source. Using KDE/X11. Kernel: 6.0.12-arch1-1

Just tried with driver versions 520.56.06 and 515.86.01. Does not seem to work on these drivers even though it did in the past.

No errors ever show, the gamescope process seems to just completely freeze.

[cchulo]

exact same setup as tsawhill, can confirm this happens on my end as well

[cchulo]

I don't understand why it just started working for me out of nowhere :(
launch options used: gamemoderun gamescope -h 1080 -r 60 -f -- %command%

Worth noting that 1080 is not my native resolution, and also worth noting that 1080 was NOT working at all yesterday

system specs:
OS: EndeavourOS
LinuxKERNEL: 6.1.1-arch1-1
CPU: AMD Ryzen 5 5600X 6-Core
GPU: NVIDIA GeForce RTX 3080
GPU DRIVER: NVIDIA 525.60.11
RAM: 32 GB

[cchulo]

new update, I got it to work consistently now at native resolution, I NEED to specify H and h flags:

gamemoderun gamescope -H 1440 -h 1440 -r 60 -f -- %command%

Edit: Forgot to mention, the previous command I posted (the one with -h 1080 -r 60 -f) works intermittently, sometimes it works, sometimes it doesn't, it especially stops working when I try to set -h 1440 without specifying -H

[tsawhill]

I can verify the above, setting both H and h flag seems to completely fix this issue for now. Thanks!

Edit: If I DO provide -W and -w flags, then it crashes in the same manner as before.
Edit 2: Even if I can get it to launch with these flags, seems to freeze the same way after a little bit of use.

[cchulo]

I am still encountering issues, as tsawhill mention in Edit2, after a few times and it starts having issues starting up again

[cchulo]

this may actually be an issue with steam

So when using gamescope with a non-steam game, i add gamescope -H 1440 -h 1440 -r 60 -f -- %command% in the launch options, and surprise surprise it works 50% of the time

However, if I use gamescope like this instead, it works 100% of the time:

Running gamescope from the CLI works flawlessly as well (I use vkcube to test this out)

[tsawhill]

This does not apply to me, I see the same freezes outside of Steam. For example, running gamescope -f glxgears will freeze. And if I pass the H,h flags it runs for about 10-15 secs then freezes.

[cchulo]

This does not apply to me, I see the same freezes outside of Steam. For example, running gamescope -f glxgears will freeze. And if I pass the H,h flags it runs for about 10-15 secs then freezes.

it works so as long as I provide H/h flags, other than that its not consistent

[cchulo]

@tsawhill the game that you are trying to launch, can you check the steamapps/shadercache for that game, and then go into fozpipelinesv6

When you attempt to load, and it fails, is there a foz file that is created that is less than 100kb? when it does succeed is the foz file that gets created larger like 147.3 kb?

I found that if the foz file that is generated is less than 100kb, it results in gamescope failing to boot up:

This screenshot is from emulationstation running with gamescope, I opened a separate issue #749 with these files accumulating, but I think there might be correlation between failing to bootup vs the size of these files.

[cchulo]

btw in my screenshot above, you can see that I launched 3 times, it failed the very first time, but succeeded the second and third time, dolphin did not update the date created, but was created 2 minutes ago (I ran the game three times in quick succession)

[halcyonhippo]

Having the same issue. Using -h + -H makes no difference. The game only launches at random. Something like 1 out of every 10 to 15 times. Gamescope seems to be completely unusable for me.

system specs:
OS: Nobara Linux 37
LinuxKERNEL: 6.1.9-200.fsync.fc37.x86_64
CPU: Intel i5-6600K
GPU: NVIDIA GeForce GTX 970
GPU DRIVER: NVIDIA 525.85.05
RAM: 32 GB

Edit: I've made some headway... it appears setting the resolution higher than a -H of about 1200 causes the launch to become unstable. At 1200 or less, it launches successfully fairly consistently. I'm not sure why it's unstable at higher resolutions. Unfortunately my whole goal of gamescope was for upscaling to 4K. Game runs at 1080p as its max res, so being able to go up to 1200 is something. Hopefully someone can figure out what's happening soon.

[luni3359]

@halcyonhippo You're probably experiencing good rng in those tests, because I still get random segfaults when opening games of any resolution.

No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
wlserver: [backend/headless/backend.c:68] Creating headless backend
vulkan: selecting physical device 'Intel(R) UHD Graphics (CML GT2)': queue family 0
vulkan: physical device supports DRM format modifiers
vulkan: supported DRM formats for sampling usage:
vulkan:   0x34325241
vulkan:   0x34325258
vulkan:   0x3231564E
wlserver: Running compositor on wayland display 'gamescope-0'
wlserver: [backend/headless/backend.c:16] Starting headless backend
wlserver: [xwayland/sockets.c:63] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use
wlserver: [xwayland/server.c:108] Starting Xwayland on :1
(EE) could not connect to wayland server
/usr/bin/prime-run: 2 行: 800774 Segmentation fault      (コアダンプ) __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia "$@"
Process exited with code 139.

[halcyonhippo]

@luni3359 it appears you were correct. Still failing intermittently at these lower resolutions.

[halcyonhippo]

Running gamescope under a wayland session seems to always succeed. The sporadic failures are only happening when using an X11 session. Is there some sort of race condition with registering the gamescope magic with X11 that's breaking things? I notice a lot of the error logs here mention something along the lines of /tmp/.X11-unix/X0: Address already in use... I'm so out of my depths with this though I don't even really know where to begin.

[LethalManBoob]

would like to clarify this is only an issue on gnome, kde is fine

[TheSaneWriter]

I experienced this same issue on the following system:
OS: EndeavourOS
LinuxKERNEL: 6.5.4-arch2-1
CPU: AMD Ryzen 5 1700X 8-Core
GPU: NVIDIA GeForce RTX 1070 Ti
GPU DRIVER: NVIDIA 525.60.11
RAM: 32 GB
Graphics PLatform: X11
KDE Plasma Version 5.27.8

Running the command gamemoderun gamescope -H 1440 -h 1440 -r 60 -f -- %command% also allows it to run at full screen consistently for me. I'm not sure why.

[zark0-UwU]

would like to clarify this is only an issue on gnome, KDE is fine

I am having a different experience, it freezes doing full screen on Kde.
as someone mentioned in this thread, it does freeze every time the window goes full screen, even when using the hotkey.
Unfortunately, my logs don't show anything as they appear to freeze as well.

My System:
OS: EndeavourOS
KDE Plasma Version: 5.27.9
KDE Frameworks Version: 5.111.0
Qt Version: 5.15.11
Kernel-Version: 6.6.1-arch1-1 (64-bit)
Graphics Platform: Wayland
CPU: AMD Ryzen 7 5800X 8-Core
GPU: NVIDIA GeForce RTX 2070 SUPER
GPU DRIVER: NVIDIA 545.29.02
RAM: 32 GB

[LethalManBoob]

Forgot to update but for a while now it doesnt work at all on nvidia kde. I cant seem to get gamescope to work at all.
Nice profile zarko btw