journal/syslog floods, exhausting disk space

[khaeru]

Describe the bug

Having GSConnect installed and enabled periodically leads to a full disk. As a result, rsyslogd takes up 100% CPU, lots of fan noise and heat, etc.

Inspecting the journal ($ journalctl) yields more than 1 million repetitions of messages like:

Aug 14 21:48:24 khaeru-desktop org.gnome.Shell.Extensions.GSConnect[7184]: == Stack trace for context 0x561a74ee80b0 ==
Aug 14 21:48:24 khaeru-desktop org.gnome.Shell.Extensions.GSConnect[7184]: #0   561a751579a8 i   /home/khaeru/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:744 (7f68dc0851f0 @ 1451)
Aug 14 21:48:24 khaeru-desktop gjs[7642]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Aug 14 21:48:24 khaeru-desktop gjs[7642]: The offending callback was SourceFunc().

…and the same four messages in /var/log/syslog. /var/log/syslog ends up filling the disk (at >10 GB); however the journal (/var/log/journal/*) is also >1 GB, probably only smaller because it's stored as binary rather than plain text.

Steps To Reproduce:

1. Install & enable GSConnect.
2. Wait.

Expected behavior
Logs are not filled with GSConnect-related error messages.
Disk space is not exhausted.

Support Log

The GSConnect settings dialog cannot be opened while the logs are flooding, and so a support log cannot be generated. It is necessary to kill the gjs process (e.g. 7642 in the above messages), after which it seems to be automatically restarted, and the dialog can be opened.

System Details (please complete the following information):

• GSConnect version: 24
  • Installed from: GNOME Extensions Website
• GNOME/Shell version: 3.32.2-2ubuntu1~ubuntu19.04.1
• Distro/Release: Ubuntu 19.04

GSConnect environment (if applicable):

• Paired Device(s): None; bug occurs when no device is paired.

[ferdnyc]

1 million, huh? That is... excessive! Wow.

Those areas of the code have been reworked quite a bit since v24 was released. (Ages ago, in gnome-shell-extension years, since it'll be 3 months old on Friday.) With a v25 release beginning to take shape on the horizon, I was wondering if you'd be willing to try installing the v25-rc3 release candidate to see if it clears the problem up?

You can find "manual" upgrade instructions for installation from zipfile on the Wiki's "Installation" page.

[khaeru]

Thanks, I will try this at the next opportunity and reply with any results.

[JamesSwift]

Hi, just to report I had the same experience today. My logs grew to over 50GB then my machine pretty much just died (also on V24). The messages were mostly:

Sep  2 00:07:27 home org.gnome.Shell.Extensions.GSConnect[32155]: == Stack trace for context 0x55a6063e60b0 ==
Sep  2 00:07:27 home gjs[32590]: The offending callback was SourceFunc().
Sep  2 00:07:27 home org.gnome.Shell.Extensions.GSConnect[32155]: #0   55a6066560d8 i   /home/james/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:744 (7f8ce50851f0 @ 1451)

[ferdnyc]

@JamesSwift , @khaeru

Now that v25 v26 is released, can you confirm whether this issue is still occurring with the updated version?

[JamesSwift]

I've had no problems since trying the beta, and now the latest release.

[ferdnyc]

Thanks @JamesSwift , glad to hear it!

I'm going to close this, if this same problem occurs (for anyone) with v26, please open a new issue with relevant log messages.

[lamyergeier]

GSConnect V37.
Ubuntu 20.04

Syslog about 50 gigabytes!

May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: == Stack trace for context 0x5595d63af1a0 ==
May  5 22:58:38 X550JX gjs[3882]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: #0   5595d64607e0 i   /home/nikhil/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:1125 (49cecf8d178 @ 1324)
May  5 22:58:38 X550JX org.gnome.Shell.Extensions.GSConnect[3882]: == Stack trace for context 0x5595d63af1a0 ==
May  5 22:58:38 X550JX gjs[3882]: The offending callback was SourceFunc().

[andyholmes]

Did you recently upgrade from Ubuntu 18.04? If so you may have to start with a fresh configuration for GSConnect. See this page in the wiki.

[dniku]

Seeing this right now under Manjaro. I have community/gnome-shell-extension-gsconnect 37-1 installed.

[finnjohnsen]

I had this problem today on Ubuntu 20.04 and it had filled up my syslog with 331gb. Full disks on linux isnt pretty and I thought my machine was dead since X wouldnt start. Had to ctrl-alt-F2 to TTY to find and delete the syslog.1 -file in order to get my (x) to boot properly.

I was about to buy a new computer, as I assumed my 4yo computer had died.

If im correct with posting this experience on this issue, which maybe not correct...?

These are the two lines repeating in my logs (331gb):
**
The offending callback was SourceFunc().
Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget

**

[elacheche]

Hello,
I just had the same issue on a family desktop.. The weird thing was that no one installed anything on it, and I never installed gsconnect on it..

After debugging, turned out that snapd was the cause root, apparently it installed multiple gnome related things by default, and probably did an auto update.. → Don't have an other explanation, the system was installed more than 6 months ago, and no one touched it since.

After disabling all snap related services and emptying the syslog, things are back to normal.

Distro/Release: Ubuntu 20.04

[daniellandau]

@finnjohnsen could you post some lines of the actual log? I'm especially interested in what is the interval at which these repeat. Are you sure that GSConnect was causing the log spam, or could it have been something else?

To the original post, one reason the journal is smaller is that it has actual settings for max storage usage as well as limits of how much at a minimum to try and keep free.

[chrisharden42]

Just had the same problem - Ubuntu 20.04.1 LTS

I dual use the computer to 1) run my TV with gnome and 2) run a nextcloud server. I can see from Prometheus that my disk started filling at ~6.30am today and I caught it after it consumed ~135gig at 10.30am - a few hundred meg away from zero disk space.

I've already truncated syslog to keep the server alive, but I've captured the following from the console as I investigated:

Syslog:
Jan 3 10:09:30 viper gjs[2613]: The offending callback was SourceFunc().
Jan 3 10:09:30 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (56ff1b8d178 @ 963)
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:09:30 viper gjs[2613]: The offending callback was SourceFunc().
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (56ff1b8d178 @ 963)
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:09:30 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Jan 3 10:09:30 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensionschrichrchris@vicchris@viper:~$ sudo tail /var/log/syslog
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:12:48 viper gjs[2613]: The offending callback was SourceFunc().
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (56ff1b8d178 @ 963)
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:12:48 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (56ff1b8d178 @ 963)
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: == Stack trace for context 0x5568c49db1a0 ==
Jan 3 10:12:48 viper gjs[2613]: The offending callback was SourceFunc().
Jan 3 10:12:48 viper org.gnome.Shell.Extensions.GSConnect[2613]: #0 5568c4a8af80 i /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (56ff1b8d178 @ 963)
Jan 3 10:12:48 viper gjs[2613]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs.

Top:
chris@viper:~$ top

top - 10:09:59 up 7 days, 12:44, 2 users, load average: 3.45, 2.81, 2.56
Tasks: 282 total, 4 running, 278 sleeping, 0 stopped, 0 zombie
%Cpu(s): 35.0 us, 48.6 sy, 0.0 ni, 16.1 id, 0.2 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 7629.2 total, 138.5 free, 1248.8 used, 6242.0 buff/cache
MiB Swap: 2048.0 total, 686.9 free, 1361.1 used. 6033.7 avail Mem
Unknown command - try 'h' for help
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
885 syslog 20 0 363616 88224 3152 R 154.5 1.1 133:43.08 rsyslogd
306 root 19 -1 543864 307864 306192 R 97.3 3.9 210:01.44 systemd+
2613 jarvis 20 0 3663072 33164 17788 R 75.1 0.4 151:10.25 gjs
884 prometh+ 20 0 1548828 65104 14132 S 5.3 0.8 31:22.05 prometh+
883 prometh+ 20 0 1445720 16052 6036 S 1.7 0.2 68:10.26 prometh+
1128915 www-data 20 0 351944 44860 34980 S 1.0 0.6 5:29.12 php-fpm+
1218559 root 20 0 1644216 38680 16704 S 1.0 0.5 1:12.83 grafana+
850 message+ 20 0 9832 6032 3592 S 0.7 0.1 17:28.12 dbus-da+
11029 www-data 20 0 787056 3704 3328 S 0.3 0.0 17:28.36 loolwsd
1269271 www-data 20 0 2242312 10132 6488 S 0.3 0.1 0:00.57 apache2
1 root 20 0 171552 11552 6876 S 0.0 0.1 11:48.21 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.21 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par+
9 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_perc+
10 root 20 0 0 0 0 S 0.0 0.0 0:15.70 ksoftir+
11 root 20 0 0 0 0 I 0.0 0.0 13:02.60 rcu_sch+

ps:

chris@viper:~$ ps ax | grep gsconnect
2613 ? Rl 154:36 gjs /home/jarvis/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js
1305553 pts/0 R+ 0:00 grep --color=auto gsconnect

Resolution:
chris@viper:~$ sudo killall gjs
chris@viper:/var/log$ sudo truncate syslog --size 0

After killing gjs and performing no other action I can't see any negative impact on the system and everything seems to be running as normal.

[daniellandau]

I'm sure the daemon probably shouldn't crash at all, but that doesn't look like enough volume to fill >100GB in a matter of a few hours. I'd very much like to help, but so far I have no good idea where to look.

[sonnyp]

Possibly related: https://gitlab.gnome.org/GNOME/polari/-/issues/148 and https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/1868

[gregharvey]

I'm also seeing this error, it's intermittent but when it happens my gnome-shell completely freezes. I am running gsconnect, I'll disable it for a little while and see if the crashing goes away.

[gregharvey]

In my case I don't think it was gsconnect, after more debugging I disabled hardware acceleration in my browser (Brave) and I haven't had a crash since...

[santiagofn]

Ubuntu 21.04 & GSConnect 45. I had the same issue as the OP (my syslog grew as much as 900 Gb). I needed to uninstall the extension, reboot my computer and install GSConnect again. After that all seems to work ok.

[gregharvey]

Weirdly, after a few months of this not happening, I'm now experiencing regular desktop freezes with this same message again. Will try removing GSConnect and see what happens...

[anedroid]

I've just experienced the same problem. My entire root partition has been filled by /var/log/syslog.

OS: Trisquel Nabia
GNOME Shell: 3.36.9
GSConnect: 53

Logs fragment:

Mar 21 14:04:36 trisquel gjs[2092]: Attempting to run a JS callback during garbage collection. This is most likely caused by destroying a Clutter actor or GTK widget with ::destroy signal connected, or using the destroy(), dispose(), or remove() vfuncs. Because it would crash the application, it has been blocked.
Mar 21 14:04:36 trisquel gjs[2092]: The offending callback was SourceFunc().

Also, this warning shows up randomly:

Mar 21 14:04:36 trisquel org.gnome.Shell.Extensions.GSConnect[2092]: == Stack trace for context 0x55698b23d1a0 ==
Mar 21 14:04:36 trisquel org.gnome.Shell.Extensions.GSConnect[2092]: #0 55698b2ecd40 i /home/***/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/daemon.js:729 (37899718d178 @ 963)