X11 Crashes on resume from suspend; suspend hangs with black screen

[avalonwilliams]

I'm running Gentoo with elogind/openrc, and since install (a couple months ago) I've had issues with suspend/resume, since setting up NVIDIA drivers. I'm on an Optimus laptop, (Thinkpad T15g gen 1 with an RTX2070), and have the proprietary drivers.

Depending on the setting of HandleNvidiaSleep and/or the presence of a script like the ones posted in #140 I will get two different behaviors. If I have HandleNvidiaSleep=yes or in my config or if I have a suspend script that calls nvidia-sleep.sh (or one does chvt and such manually), my system will hang indefinetly with a black screen (backlight is still on, however, and system hasn't yet entered sleep state from what the kernel logs say). However, if I do not have any settings for handling NVIDIA sleep, I get a black screen for around 20-30 seconds, and then it goes to sleep; when my laptop wakes up, my X server segfaults, with the Xorg.0.log showing this:

[  6961.019] (II) systemd-logind: releasing fd for 13:89
[ 11729.685] (EE) intel(0): Failed to set backlight intel_backlight for output eDP1, disabling
[ 23989.595] (**) Option "fd" "29"
[ 23989.595] (II) event2  - Power Button: device removed
[ 23989.595] (**) Option "fd" "26"
[ 23989.595] (II) event7  - Video Bus: device removed
[ 23989.595] (**) Option "fd" "30"
[ 23989.595] (II) event6  - Video Bus: device removed
[ 23989.595] (**) Option "fd" "32"
[ 23989.595] (II) event0  - Sleep Button: device removed
[ 23989.595] (**) Option "fd" "31"
[ 23989.595] (II) event10 - Integrated Camera: Integrated C: device removed
[ 23989.595] (**) Option "fd" "28"
[ 23989.595] (II) event3  - AT Translated Set 2 keyboard: device removed
[ 23989.595] (**) Option "fd" "53"
[ 23989.595] (II) event5  - TPPS/2 Elan TrackPoint: device removed
[ 23989.595] (**) Option "fd" "27"
[ 23989.595] (II) event8  - ThinkPad Extra Buttons: device removed
[ 24014.628] (EE) systemd-logind: failed to ack pause: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
[ 24039.654] (EE) systemd-logind: failed to ack pause: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
[ 24044.120] (EE)
[ 24044.120] (EE) Backtrace:
[ 24044.120] (EE) 0: /usr/bin/X (xorg_backtrace+0x5b) [0x56359911785b]
[ 24044.121] (EE) 1: /usr/bin/X (0x563598fde000+0x13d495) [0x56359911b495]
[ 24044.121] (EE) 2: /lib64/libc.so.6 (0x7fea71877000+0x3d7c0) [0x7fea718b47c0]
[ 24044.121] (EE) 3: /lib64/libc.so.6 (0x7fea71877000+0x16d35a) [0x7fea719e435a]
[ 24044.121] (EE) 4: /usr/bin/X (0x563598fde000+0x18faf1) [0x56359916daf1]
[ 24044.121] (EE) 5: /usr/lib64/libdbus-1.so.3 (dbus_connection_dispatch+0x3bd) [0x7fea71adc61d]
[ 24044.121] (EE) 6: /usr/lib64/libdbus-1.so.3 (0x7fea71ac5000+0x17a28) [0x7fea71adca28]
[ 24044.121] (EE) 7: /usr/bin/X (0x563598fde000+0x1b0051) [0x56359918e051]
[ 24044.122] (EE) 8: /usr/bin/X (0x563598fde000+0x13de01) [0x56359911be01]
[ 24044.122] (EE) 9: /usr/bin/X (WaitForSomething+0x193) [0x563599115053]
[ 24044.122] (EE) 10: /usr/bin/X (0x563598fde000+0x72d44) [0x563599050d44]
[ 24044.122] (EE) 11: /usr/bin/X (0x563598fde000+0x76cab) [0x563599054cab]
[ 24044.122] (EE) 12: /lib64/libc.so.6 (0x7fea71877000+0x2931a) [0x7fea718a031a]
[ 24044.122] (EE) 13: /lib64/libc.so.6 (__libc_start_main+0x7c) [0x7fea718a03cc]
[ 24044.122] (EE) 14: /usr/bin/X (_start+0x21) [0x56359901a051]
[ 24044.122] (EE)
[ 24044.123] (EE) Segmentation fault at address 0x0
[ 24044.123] (EE)
Fatal server error:
[ 24044.123] (EE) Caught signal 11 (Segmentation fault). Server aborting
[ 24044.123] (EE)
[ 24044.123] (EE)
Please consult the The X.Org Foundation support
	 at http://wiki.x.org
 for help.
[ 24044.123] (EE) Please also check the log file at "/home/avalon/usr/share/xorg/Xorg.0.log" for additional information.
[ 24044.123] (EE)
[ 24045.059] (EE) Server terminated with error (1). Closing log file.

From some debugging/logging done with a hook script, I managed to figure out that the indefinite hanging I experience happens when changing VTs.

These problems occur regardless of the setting of SuspendMode (I tried setting it to deep to see if that would help, it didn't), and otherwise I have default configs.

I'm on elogind version 246.10-r2, kernel version 5.15.48-gentoo-dist, and NVIDIA driver version 510.73.05-r1, though it should be noted that these issues have persisted across updates of all three of these, as well as updates of X.

No display manager is used, I just run startx in my .profile

Also, manually suspending via echo mem > /sys/power/state works fine, as does running loginctl suspend from a TTY.

Xorg.0.log

emerge --info output

Possibly related to #140

[yancelawang]

Up....
I also having the same problem and my laptop also optimus laptop and use nvidia driver.

[dymn]

Confirm that.

I have installed Gnome (openrc), nvidia-drivers-515.57 (tried also 510.73.05-r1).

With gdm and Gnome (wayland) and Gnome (X11):

1. with current stable kernel in Gentoo 5.15.52 the system can't suspend, but hang with black screen, exit only with REISUB.
2. with kernel 5.18.14 the system going to suspend, but can't resume - it hangs with black screen and exit only with REISUB.
3. Gnome with sddm - the same.

I didn't find anything in the logs that clearly indicates the reason

Sway from git + sddm - the same behavior.

Openbox with gdm - the same.

Old system with Openbox and sddm - all is OK.

I tried suspend with echo mem > /sys/power/state, but the system does not suspend, immediately returns to Gnome. I don't remember exactly, but it seems with the error "broken pipe".

[p5k369]

I can also confirm that.
with kernel 5.15.59-gentoo and nvidia-drivers-510.85.02.
Suspending works and resuming also, until X should show up. Login in via ssh and restart the display-manager will make the machine usable again. However it is a bit inconvenient to have an additional device ready if you want to resume.

I am using latest stable gnome desktop with gdm on xorg-21.1.4.

Suspending with echo mem > /sys/power/state and resuming works flawlessly.

Would any logs be helpful?

[Ermiq]

I have the same issue as avalonwilliams described when I don't use a display manager and start X session with xinit or startx on my NVidia/Intel hybrid laptop.
If I remove NVidia driver then suspend/resume works fine with nouveau driver. With NVidia driver loaded it doesn't. If I install lightdm then everything works fine.
So, in my case it seems to happen when I don't use a DM and use NVidia driver.
The problem exists on Devuan Chimaera (kernel 5.10.08, nvidia driver 470.129, sysVinit), Artix Linux (kernel 5.16.16, nvidia driver 510.54, runit).
On Devuan I have system hang up with a black screen when I open laptop's lid after suspension.
On Artix I have a black screen for ~20 seconds, then laptop goes to suspend mode, when I open up the lid X server crashes with the same error messages in Xorg logs posted by avalonwilliams.

[avalonwilliams]

I have the same issue as avalonwilliams described when I don't use a display manager and start X session with xinit or startx on my NVidia/Intel hybrid laptop. If I remove NVidia driver then suspend/resume works fine with nouveau driver. With NVidia driver loaded it doesn't. If I install lightdm then everything works fine. So, in my case it seems to happen when I don't use a DM and use NVidia driver. The problem exists on Devuan Chimaera (kernel 5.10.08, nvidia driver 470.129, sysVinit), Artix Linux (kernel 5.16.16, nvidia driver 510.54, runit). On Devuan I have system hang up with a black screen when I open laptop's lid after suspension. On Artix I have a black screen for ~20 seconds, then laptop goes to suspend mode, when I open up the lid X server crashes with the same error messages in Xorg logs posted by avalonwilliams.

Can confirm that the issue only occurs without a display manager. Using lxdm works fine on my machine.

[lukolszewski]

For me the issue occurs with gdm(not just with no DM). I run elogind 246.10 on (non)guix. Nvidia driver 515.65.01, kernel 5.18.18, gdm and xfce. With HandleNvidiaSleep=no machine suspends fine, but on resume it hangs with the cursor visible on the left top of the screen. With HandleNvidiaSleep=yes it suspends fine, hangs on black screen on resume.

No errors are logged at all.

If I switch to a text console and use loginctl suspend it suspends and resumes correctly. Then after resume I can switch to xorg again and it works fine.

I too tested it with deep sleep with no difference.

[avalonwilliams]

For me the issue occurs with gdm(not just with no DM). I run elogind 246.10 on (non)guix. Nvidia driver 515.65.01, kernel 5.18.18, gdm and xfce. With HandleNvidiaSleep=no machine suspends fine, but on resume it hangs with the cursor visible on the left top of the screen. With HandleNvidiaSleep=yes it suspends fine, hangs on black screen on resume.

No errors are logged at all.

If I switch to a text console and use loginctl suspend it suspends and resumes correctly. Then after resume I can switch to xorg again and it works fine.

I too tested it with deep sleep with no difference.

Could you post the output of loginctl show-session self on a logged in XFCE session. Also try to see if lightdm works.

[yancelawang]

Here my loginctl show-session self. I'm use i3wm:

Id=2 User=1000 Name=yance Timestamp=Tue 2022-09-06 23:43:02 WITA TimestampMonotonic=35477039 VTNr=1 Seat=seat0 TTY=tty1 Remote=no Service=login Leader=6506 Audit=2 Type=tty Class=user Active=yes State=active IdleHint=no IdleSinceHint=1662478977150000 IdleSinceHintMonotonic=30457989 LockedHint=no

[avalonwilliams]

Here my loginctl show-session self. I'm use i3wm:

Id=2 User=1000 Name=yance Timestamp=Tue 2022-09-06 23:43:02 WITA TimestampMonotonic=35477039 VTNr=1 Seat=seat0 TTY=tty1 Remote=no Service=login Leader=6506 Audit=2 Type=tty Class=user Active=yes State=active IdleHint=no IdleSinceHint=1662478977150000 IdleSinceHintMonotonic=30457989 LockedHint=no

As I suspected, it seems like this bug mostly occurs when elogind has the session type=TTY

Most likely this stems from the suspend code falsely assuming that anything with type=TTY is a terminal session, when startx, Wayland compositors, and seemingly GDM sessions do not have this set correctly.

For now the solution seems like to use LightDM or LXDM, but to use those it seems like you have to run X as root, which kind of defeats half of the purpose of logind.

[DarkDefender]

I have this issue also on two computers. One is a laptop with an nvidia quadro card and on other is a regular desktop with a 1080 ti. I have two other computers with AMD GPUs running the same version of elogind and the library versions are exactly the same as on the nvidia computers. There I do not have any issues with suspend.

As others have mentioned the echo mem > /sys/power/state seems to work with the nvidia gpus.
To see if having a session with a type that wasn't TTY, I launched sway which does set the type to wayland.

In sway when I type loginctl suspend, the system seems to try to suspend but dmesg reports that is trying to suspend, but nothing seems to happen. Nothing visual indicates that it has tried to suspend and I seemingly can continue to use the computer. After this echo mem > /sys/power/state fails as the device is busy.

If I then try to reboot or power off the computer with the reboot or poweroff command, the computer goes to sleep.
When I wake if from sleep I get back into sway but the computer then reboots/shuts off (because it was queued up).

I've tried with and without the HandleNvidiaSleep, doesn't seem to make any difference.

[Ftonans]

I had quite similar problem but solved it on Artix by editing /lib64/elogind/system-sleep/nvidia

#!/bin/sh
case "${1-}" in
    'pre')
        /usr/bin/nvidia-sleep.sh suspend
        ;;
    'post')
        /usr/bin/nvidia-sleep.sh resume &
        ;;
    *)
        exit 1
        ;;
esac

[Yamakuzure]

Interesting... I have an Optimus Laptop, too, sporting a Nvidia T2000, and hibernate/suspend + wakeup/resum works just fine.

Let's investigate this once the next release is at least at rc level.

[yancelawang]

I had quite similar problem but solved it on Artix by editing /lib64/elogind/system-sleep/nvidia

#!/bin/sh
case "${1-}" in
    'pre')
        /usr/bin/nvidia-sleep.sh suspend
        ;;
    'post')
        /usr/bin/nvidia-sleep.sh resume &
        ;;
    *)
        exit 1
        ;;
esac

I was change like yours, but still not solved.

[yancelawang]

Interesting... I have an Optimus Laptop, too, sporting a Nvidia T2000, and hibernate/suspend + wakeup/resum works just fine.

Let's investigate this once the next release is at least at rc level.

Interesting. Can you share your setup?

[Yamakuzure]

Gentoo Linux running openrc+elogind with Plasma/X11, wayland still does not work well with nvidia + external monitors, on a Dell Precision M7550 with 10th gen i7, 64GB RAM, a bunch of 5 external hard drives, video editing needs a lot of space, and a nvidia Quadro T2000 alongside the regular Intel HD.

I do neither use a hook script to call nvidia-sleep.sh nor do I let elogind handle vt switches (HandleNvidiaSleep is commented out and defaults to 'no').

Here is what I have configured for Optimus:

1. Grub is set up to add modprobe.blacklist=nvidia,nouveau to the kernel command line
2. /etc/modules-load.d/autoload.conf has nvidia_uvm in it, which always should be first on Optimus systems
3. I have nvidia-persistencedin runlevel 'default' - although I have to admit that I do not know whether this is relevant

Oh! And I also have this in my ~/.xinitrc:

#!/bin/sh

xrandr --setprovideroutputsource modesetting NVIDIA-0
xrandr --auto

I did several suspend-resume-cycles in the last 2 hours while testing v252_rc2, and nvidia-smi still shows 46 entries.

What else do you want to know?

[fabian-thomas]

I experience similar issues. For me both suspend and hibernate work if I call nvidia-sleep.sh with a system-sleep script. However when I enable HandleNvidiaSleep, Xorg crashes.

One further issue I noticed when using the nvidia-sleep.sh script is that suspend-then-hibernate does not work perfectly. After coming from hibernation, the vt is not switched back and stays at 63. That is because the nvidia-sleep.sh script is called with hibernate when coming from suspend, which then overwrites the saved vt. Hardcoding the vt value to be 1 breaks the nvidia-sleep.sh script since then we hibernate sometimes while on vt 1 (because resume in the nvidia-sleep script is triggered in a bad moment I guess). This could be fixed if elogind would provide the system-sleep scripts more information what will happen. For example hibernate-after-suspend instead of only hibernate in the case of suspend-then-hibernate. Then one could patch the nvidia script so that it does not switch vts when going from suspend to hibernate.

[yancelawang]

Gentoo Linux running openrc+elogind with Plasma/X11, wayland still does not work well with nvidia + external monitors, on a Dell Precision M7550 with 10th gen i7, 64GB RAM, a bunch of 5 external hard drives, video editing needs a lot of space, and a nvidia Quadro T2000 alongside the regular Intel HD.

I do neither use a hook script to call nvidia-sleep.sh nor do I let elogind handle vt switches (HandleNvidiaSleep is commented out and defaults to 'no').

Here is what I have configured for Optimus:

1. Grub is set up to add `modprobe.blacklist=nvidia,nouveau` to the kernel command line

2. `/etc/modules-load.d/autoload.conf` has `nvidia_uvm` in it, which always should be first on Optimus systems

3. I have `nvidia-persistenced`in runlevel 'default' - although I have to admit that I do not know whether this is relevant

Oh! And I also have this in my ~/.xinitrc:

#!/bin/sh

xrandr --setprovideroutputsource modesetting NVIDIA-0
xrandr --auto

I did several suspend-resume-cycles in the last 2 hours while testing v252_rc2, and nvidia-smi still shows 46 entries.

What else do you want to know?

Thank You. This seems to only happen when not using the display manager. I really miss suspend being able to work on my laptop again. For a while, i didn't use elogind to suspend my laptop.

[Yamakuzure]

This is deemed to be on hold until the next release is out and the issue can be reproduced with it.

[rsauex]

I'm experiencing the same issue on Guix running Elogind 252.9, Xorg 21.1.4, and Nvidia driver 515.76. I'm using ThinkPad p15 gen 2 with RTX A2000 Mobile.

Here is what I discovered after some digging:

Somewhere during suspend the system switches to VT 63. For me it happens even when suspending manually using echo 'mem' > /sys/power/state, so I suspect it's an Nvidia driver's thing:

root@gamma ~# date
Sun 10 Sep 2023 20:24:36 EEST
root@gamma ~# echo 'mem' > /sys/power/state
root@gamma ~# cat /var/log/debug | grep 'VT changed'
Sep 10 20:24:43 localhost elogind[1076]: VT changed to 63
Sep 10 20:25:05 localhost elogind[1076]: VT changed to 1

When the switch happens, Xorg takes action (xf86VTSwitch) and ultimately calls systemd_logind_ack_pause that calls elogin's PauseDeviceComplete method over dbus and waits for a response:

root@gamma ~# cat /var/log/debug | grep 'PauseDeviceComplete'
Sep 10 20:24:43 localhost elogind[1076]: Got message type=method_call sender=:1.17 destination=org.freedesktop.login1 path=/org/freedesktop/login1/session/c2 interface=org.freedesktop.login1.Session member=PauseDeviceComplete cookie=31 reply_cookie=0 signature=uu error-name=n/a error-message=n/a

When suspending manually that's not a problem and elogind happily sends a response, and VT switch completes successfully:

root@gamma ~# cat /var/log/debug | grep -A 2 'PauseDeviceComplete'
...
Sep 10 20:24:43 localhost elogind[1076]: Sent message type=method_return sender=n/a destination=:1.17 path=n/a interface=n/a member=n/a cookie=348 reply_cookie=31 signature=n/a error-name=n/a error-message=n/a
Sep 10 20:24:43 localhost elogind[1076]: VT changed to 63

However, that's not the case when suspending using elogind. When Xorg sends the message, elogind is stuck writing to /sys/power/state and can't receive the message. The VT switch doesn't finish and Xorg crashes on resume with:

[   551.811] (EE) systemd-logind: failed to ack pause: Did not receive a reply.

I've tried to check whether that's the case by applying a patch that makes elogind write to /sys/power/state in a separate thread, and now it works correctly. This is definitely not the correct way to fix it, though... It surely breaks some other things.

Full log for the manual suspend:
elogind-manual.txt

Full log with the patch applied:
elogind-success.txt

Full log without the patch:
elogind-failure.txt

I'd be glad to provide more info

[DarkDefender]

AWESOME!

The patch also seems to fix the suspend issue for me!
I'll try it on a few different computers just to see if it also solves the nvidia suspend issues on those as well.

[DarkDefender]

The patch seems to have solved the suspend issues on all nvidia computers I have in the computer lab (I tested 5 of them).

All works well now in Xorg and I can suspend in Wayland now as well. However in Wayland I get graphical corruption when resuming from suspend. But I guess this could be solved if the "HandleNvidiaSleep=yes" and "NVreg_PreserveVideoMemoryAllocations=1" was working (which they are not even with the patch sadly).

[lumpsoid]

Linux 6.6.1-artix1-1
elogind 252.9-2
nvidia 545.29.02-4
starting up Xorg with startx
DM dwm
/lib/elogind/system-sleep/nvidia modified as posted here

experiencing same thing with loginctl suspend and nvidia driver
and my findings point to the same area as @rsauex's digs
that for some reason the process is stuck on changing VT 63 (from /usr/bin/nvidia-sleep.sh)

but I didn't debug what is happening in this process after VT changes

[VPaulV]

sys-auth/elogind 246.10-r3, still having this issue with wayland+kde+nvidia. In X11 everything works without any problem
UPD: so far patch worked well, everything seems to work fine except of the NetworkManager, but that is not related to this issue I guess. Thank you @rsauex

[Yamakuzure]

sys-auth/elogind 246.10-r3, still having this issue with wayland+kde+nvidia. In X11 everything works without any problem UPD: so far patch worked well, everything seems to work fine except of the NetworkManager, but that is not related to this issue I guess. Thank you @rsauex

246.10 is very old, but glad to hear that it works.
Wayland + nvidia is still a gamble. My hopes are on KDE 6, as I never got a stable Wayland experience with KDE 5 running.

[Yamakuzure]

I've tried to check whether that's the case by applying a patch that makes elogind write to /sys/power/state in a separate thread, and now it works correctly. This is definitely not the correct way to fix it, though... It surely breaks some other things.

On the contrary, you may have found the root issue!

Although elogind is an enhanced version of systemd-logind, the sleep component integrates systemd-sleep.
This means while systemd starts systemd-sleep as a forked process, elogind does it as a function in its main thread.

So the correct method, if not forking out another process, would be to start a separate thread, because elogind can not react to dbus messages it sends to itself in the main thread and await an answer from itself, can it?

In other words, your idea could finally eradicate these suspend-resume issues pestering us for years.

Thank you very much!

[Yamakuzure]

The issue should be fixed, but I can not test it properly.
My nvidia Quadro RTX 5000 suffers from the bug shown here, which means that, while now everything looks like it is working fine, my laptop sort of freezes after kernel: PM: suspend entry (deep) event. (Have to REISUB out.)

I never had any problems with suspend resume loading the module with NVreg_PreserveVideoMemoryAllocations=0 and configuring nvidia with HandleNvidiaSleep=no, but I would have really like to properly test @rsauex fix.

[trevarj]

Just noting that on Gentoo with elogind 252.9 with a GTX 1060 it is still required to have this script placed in /usr/lib64/elogind/system-sleep/20-nvidia:

#!/bin/sh
case "${1-}" in
    'pre')
        /usr/bin/nvidia-sleep.sh suspend
        ;;
    'post')
        /usr/bin/nvidia-sleep.sh resume &
        ;;
    *)
        exit 1
        ;;
esac

[takano-akio]

I've had a similar issue (for years actually, only today I realized it was related to elogind), and elogind-252.23 works perfectly on 2 of 2 affected machines I have. Thank you!