dbus-broker leaks memory over time #201
Comments
|
Thanks a lot for the report! I have some more questions, though:
What do you mean with
Yeah, the logs are quite expressive. First of all, since dbus-broker-20, the log-spam with the following message is now suppressed: So the first 13k lines of that log can be ignored. Starting at line ...and this then continues with many more messages just like the last one. What is going on here is that your UID (which I assume is The first log-message in that log-excerpt tells you that the user with UID 1000 has so many pending messages (or other D-Bus objects) that it exceeded its own allowance. Hence, further attempts by that user to send more messages are rejected. Of course, I cannot tell for sure, but my assumption based on these log-messages is that dbus-broker does exactly what it is told: It tries to transmit the messages it is given by the clients. The problem is, there seems to be some client on your system that runs under your UID which either does not dequeue its messages, or exhibits some other buggy client-behavior. In my opinion, there is little Anyway, maybe you have a suspicion about which application triggers this behavior? In this case it should be a lot easier to track down. Thanks |
I just notice it has increased by small amounts whenever I occasionally check
This would be super helpful for tracking down whatever application is going nuts.
No, I don't, sorry. I'm not even sure where to start. :/ |
|
I think I'm having the same issue. Fedora 30 |
|
What is hitori? Am I reading correctly, that you have a process (hitori) that is consuming huge amounts of memory, and while it is dbus-broker is steadily consuming more. Then hitori stops consuming memory and dbus stops increasing its usage (but does not go down either)? What UID is the other process running under? |
|
I notice that in both the reports the number of clients exceed 166k (and the error seems to trigger with about the same peer id). That seems like something worth investigating. Could either of you do a |
|
Unfortunately I can't provide you a |
|
It's back after Fedora 31 upgrade. busctl --no-pager --verbose
|
|
My |
|
My Gobbling up almost 2GB of RAM seems pretty hoggish and unreasonable for a broker. Is anything being done about this? As an aside, can I restart it on a running system, with desktop users logged in and it will regain it's state or will restarting it wreak havoc? |
|
I've upgraded another laptop to Fedora 31 (from Fedora 29) and it has the same issue. |
|
The new dbus-broker-v22 release now contains a debugging mode for this. In case you see this issue on a machine with v22 or newer, please run this: This dumps the entire accounting-information of dbus-broker. This will allow us to debug the situation. Don't forget to dump the process-list as well: |
|
I am closing this, as there seems to be no progress on this. dbus-broker-22 provides debugging interfaces for this. If anyone can reproduce this, please see my previous comment for details on extracting the required information. |
|
What would be the way to interprete the |
The In other words: it is a much lighter version of a core-dump, only containing the data that we need. It should be treated with the same confidentiality as a core-dump. |
|
I'm on version v24 and seeing the same issue, dbus-broker is currently consuming a bit over 270mb, I'm annexing the output of |
|
Thanks for the data and report! Your process log says that the user instance is apparently at 270MiB, but your dbus query was sent to the system instance. Can you try this again, but using In |
|
|
Right, it is |
|
@dvdhrm sorry for the ping, but I'm seeing similar behaviour with openQA (an automated test system we use extensively in Fedora) causing tests to fail occasionally. I've filed an issue at https://progress.opensuse.org/issues/90872 with the dbus state dump and ps output attached, though not during the error state as it's not currently in it (I figured the state might still show the problem building up, though). If you have any tips on how we could further debug it, that'd be great :) |
|
@AdamWill Sure! Lets move discussion there! |
|
@AdamWill FYI, I summarized how we found the resource leak in https://dvdhrm.github.io/2021/04/14/locating-dbus-resource-leaks/ |
Distro is Arch Linux.
dbus-broker version is 20, and is from the offical repo.
Usually, it's very slow and steady.
A couple of instances, I've left my machine overnight, come back, and I've only noticed when I try starting Steam, with it crashing due to a DBus error I can't recall, and then seeing that dbus-broker was consuming many GiB of RAM.
I'll try and grab screenshots next time, if necessary.
I did grab the log from
journalctl, however:dbus-broker.tar.gz
Halfway, the log suffers ridiculous spamming which is probably what resulted in the event of dbus-broker consuming GiBs of RAM.
Current dbus-broker RAM usage is 43KiB. Was certainly a lot smaller earlier today.
The text was updated successfully, but these errors were encountered: