Gamescope cannot be used to launch Steam games

[aldrenean]

Describe the bug

Attempting to use gamescope with Steam games, by adding 'gamescope -- %command%' to the game's launch options, doesn't work.

Steps To Reproduce

Steps to reproduce the behavior:

1. Install steam (programs.steam.enable = true)
2. Enable gamescope with 'programs.gamescope.enable = true'
3. Also set 'programs.gamescope.capSysNice = true'
4. Add 'gamescope -- %command%' to the launch options in the game's properties
5. Try launching the game

Expected behavior

The game should open, running inside of a gamescope instance.

Additional context

Logging the output with 'gamescope -- %command% >> out.txt' gives only one line of output: "failed to inherit capabilities: Operation not permitted". It sounds like a permissions issue, though setting capSysNice to true for gamescope should hypothetically let it renice itself.

This comment suggests that using the flatpak version of Steam resolves this issue. I have not tested this, as it's non-trivial to transition from a native to flatpak steam installation, but regardless, the behavior with the native package should be fixed.

Notify maintainers

@K900
@zhaofengli
@Scrumplex
@PedroHLC
@nrdxp

Metadata

• system: "x86_64-linux"
• host os: Linux 6.6.56, NixOS, 24.05 (Uakari), 24.05.5709.c0b1da36f7c3
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.18.8
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[K900]

It's not really possible with the way fhsenvs work currently.

[nrdxp]

also the ticket name isn't exactly correct. I use gamescope as a window manager just fine to launch steam. So what you are more saying here is that you cannot launch gamescope in a nested session

[PedroHLC]

also the ticket name isn't exactly correct. I use gamescope as a window manager just fine to launch steam. So what you are more saying here is that you cannot launch gamescope in a nested session

It's even more narrow than that, you can't open it, nested, with capSysNice = true, due to the FHS bubblewrap.

[aidanmaney]

I have this issue on recent builds but not builds on/before Oct 14 (without capSysNice set on any of them). I am using the gamescope package from unstable but the program version is the same across all of them: 3.15.11.

Metadata

• system: "x86_64-linux"
• host os: Linux 6.6.52, NixOS, 24.11 (Vicuna), 24.11.20240923.30439d9
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.18.7
• channels(root): "nixos-24.05, nixpkgs-unstable"
• nixpkgs: /home/aidan/.nix-defexpr/channels/nixpkgs

[K900]

That exact issue should be impossible without capSysNice. Can you post the exact logs you're getting?

[aldrenean]

When running without capSysNice, I get the following relevant output in the console:

wlserver: Running compositor on wayland display 'gamescope-0'
wlserver: [backend/headless/backend.c:17] Starting headless backend
wlserver: Successfully initialized libei for input emulation!
wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use
wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X1: Address already in use
wlserver: [xwayland/server.c:107] Starting Xwayland on :2
ERROR: ld.so: object '/home/aldrenean/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
Xwayland: symbol lookup error: /nix/store/h8x5zjpgicr5y1qd7qv917vq1pi37401-libkrb5-1.21.3/lib/libcom_err.so.3: undefined symbol: k5_os_mutex_destroy, version krb5support_0_MIT
wlserver: [xwayland/server.c:269] Xwayland startup failed, not setting up xwm

Steam says the game is running but the only process I can see is the full gamescope command expanded to include the calls to steam-launch-wrapper, reaper, etc., which needs to be manually killed -- clicking Stop in Steam has worked a couple times but generally it hangs in "stopping" indefinitely.

[aidanmaney]

That exact issue should be impossible without capSysNice. Can you post the exact logs you're getting?

I confirmed it launches fine without gamescope. Using these launch options — LD_PRELOAD="" gamescope --backend wayland -W 3840 -H 2160 -w 3840 -h 2160 -b -e -s 1.67 --force-grab-cursor -r 144 --expose-wayland --adaptive-sync -- %command% — I get the following output:

Fossilize INFO: Setting autogroup scheduling.
pressure-vessel-wrap[8172]: W: "/run/current-system/sw/bin/getent" is unlikely to appear in "/run/host"
pressure-vessel-wrap[8172]: W: Found more than one possible libdrm data directory from provider
fsync: up and running.
Fossilize INFO: Setting autogroup scheduling.
Fossilize INFO: Setting autogroup scheduling.
chdir "/home/aidan/.local/share/Steam/steamapps/common/Deadlock"
ERROR: ld.so: object '/home/aidan/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
Game Recording - would start recording game 1422450, but recording for this game is disabled
Adding process 8432 for gameID 1422450
*** stack smashing detected ***: terminated
Uploaded AppInterfaceStats to Steam
Game Recording - game stopped [gameid=1422450]
Removing process 8432 for gameID 1422450

[K900]

OK, one of those issues is known and I'm looking into a proper fix in the next few days. For now, try removing mangohud and/or gamemode from your extraPkgs.

[K900]

Please test #351928.

[aidanmaney]

I get further than before with #351928. Instead of games not starting I get the initial launch screen (in this case the Valve logo) with the game cursor but then the gamescope window stops responding and goes black until I kill it. Here are my logs:

pressure-vessel-wrap[10067]: W: "/run/current-system/sw/bin/getent" is unlikely to appear in "/run/host"
pressure-vessel-wrap[10067]: W: Found more than one possible libdrm data directory from provider
fsync: up and running.
chdir "/home/aidan/.local/share/Steam/steamapps/common/Deadlock"
ERROR: ld.so: object '/home/aidan/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
Game Recording - would start recording game 1422450, but recording for this game is disabled
Adding process 10284 for gameID 1422450
[gamescope] [Info]  console: gamescope version 3.15.9 (gcc 13.3.0)
No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
[gamescope] [Info]  vulkan: selecting physical device 'AMD Radeon RX 6650 XT (RADV NAVI23)': queue family 1 (general queue family 0)
[gamescope] [Info]  vulkan: physical device supports DRM format modifiers
Fossilize INFO: Overriding serialization path: "/home/aidan/.local/share/Steam/steamapps/shadercache/1422450/fozpipelinesv6/steamapprun_pipeline_cache".
[gamescope] [Info]  wlserver: [backend/headless/backend.c:67] Creating headless backend
[gamescope] [Info]  xdg_backend: Seat name: seat0
[gamescope] [Info]  vulkan: supported DRM formats for sampling usage:
[gamescope] [Info]  vulkan:   AR24 (0x34325241)
[gamescope] [Info]  vulkan:   XR24 (0x34325258)
[gamescope] [Info]  vulkan:   AB24 (0x34324241)
[gamescope] [Info]  vulkan:   XB24 (0x34324258)
[gamescope] [Info]  vulkan:   RG16 (0x36314752)
[gamescope] [Info]  vulkan:   NV12 (0x3231564E)
[gamescope] [Info]  vulkan:   AB4H (0x48344241)
[gamescope] [Info]  vulkan:   XB4H (0x48344258)
[gamescope] [Info]  vulkan:   AB30 (0x30334241)
[gamescope] [Info]  vulkan:   XB30 (0x30334258)
[gamescope] [Info]  vulkan:   AR30 (0x30335241)
[gamescope] [Info]  vulkan:   XR30 (0x30335258)
[gamescope] [Info]  wlserver: Using explicit sync when available
[gamescope] [Info]  wlserver: Running compositor on wayland display 'gamescope-0'
[gamescope] [Info]  wlserver: [backend/headless/backend.c:17] Starting headless backend
[gamescope] [Info]  wlserver: Successfully initialized libei for input emulation!
[gamescope] [Error] wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use
[gamescope] [Error] wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X1: Address already in use
[gamescope] [Info]  wlserver: [xwayland/server.c:107] Starting Xwayland on :2
[gamescope] [Info]  pipewire: stream state changed: connecting
[gamescope] [Info]  pipewire: stream state changed: paused
[gamescope] [Info]  pipewire: stream available on node ID: 74
[gamescope] [Info]  xwm: Embedded, no cursor set. Using left_ptr by default.
[gamescope] [Info]  vblank: Using timerfd.
[gamescope] [Error] xdg_backend: Failed to map keymap fd.
[gamescope] [Info]  edid: Patching res 800x1280 -> 3840x2160
pressure-vessel-wrap[10314]: W: "/run/current-system/sw/bin/getent" is unlikely to appear in "/run/host"
pressure-vessel-wrap[10314]: W: Found more than one possible libdrm data directory from provider
pressure-vessel-wrap[10314]: W: Extra environment variable WAYLAND_DISPLAY set during container setup but not in known_vars; check logic
fsync: up and running.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
Setting breakpad minidump AppID = 1422450
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561198357593009 [API loaded no]
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
Fossilize INFO: Overriding serialization path: "/home/aidan/.local/share/Steam/steamapps/shadercache/1422450/fozpipelinesv6/steamapprun_pipeline_cache".
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
Fossilize INFO: Overriding serialization path: "/home/aidan/.local/share/Steam/steamapps/shadercache/1422450/fozpipelinesv6/steamapprun_pipeline_cache".
[gamescope] [Warn]  xwm: got the same buffer committed twice, ignoring.
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning:          Unsupported maximum keycode 708, clipping.
>                   X11 cannot support keycodes above 255.
> Warning:          Could not resolve keysym XF86KbdInputAssistPrevgrou
> Warning:          Could not resolve keysym XF86KbdInputAssistNextgrou
Errors from xkbcomp are not fatal to the X server
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option vk_xwayland_wait_ready overridden by environment.
[gamescope] [Warn]  xwm: got the same buffer committed twice, ignoring.
[gamescope] [Warn]  xwm: got the same buffer committed twice, ignoring.
[gamescope] [Info]  wlserver: [wayland] error in client communication (pid 10284)
[gamescope] [Info]  wlserver: [xwayland/server.c:217] Restarting Xwayland
XWAYLAND: wp_linux_drm_syncobj_manager_v1#9: error 0: wp_linux_drm_syncobj_surface_v1 already created for this surface
(EE) failed to dispatch Wayland events: Protocol error
[gamescope] [Info]  wlserver: [xwayland/server.c:107] Starting Xwayland on :2
X connection to :2 broken (explicit kill or server shutdown).
X connection to :2 broken (explicit kill or server shutdown).
X connection to :2 broken (explicit kill or server shutdown).
X connection to :2 broken (explicit kill or server shutdown).
X connection to :2 broken (explicit kill or server shutdown).
[gamescope] [Error] xwm: X11 I/O error
[gamescopereaper] [Info]  reaper: Parent of gamescopereaper was killed. Killing children.
(EE) could not connect to wayland server

[K900]

This sounds like an xwayland crash? Are you using Steam from the same nixpkgs version as the rest of your system?

[aidanmaney]

This sounds like an xwayland crash? Are you using Steam from the same nixpkgs version as the rest of your system?

Yes, I rebuilt with all packages pointed at "github:K900/nixpkgs?ref=awful-steamage"