Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to ignore oom_score completely #140

Closed
AbsurdlySuspicious opened this issue Aug 7, 2019 · 13 comments
Closed

Option to ignore oom_score completely #140

AbsurdlySuspicious opened this issue Aug 7, 2019 · 13 comments

Comments

@AbsurdlySuspicious
Copy link

AbsurdlySuspicious commented Aug 7, 2019

In my case there's a lot of memory-eager applications which sets themselves oom_score_adj=-1000 and therefore I keep getting xfce4-notifyd (with 25M VmRSS) killed first which occasionally gets an oom_score value of 1 instead of the process that actually ate all of my ram and triggered -m limits (but keeps its oom_score at 0).
I don't really know is that a common problem but anyway, in my opinion it'll be good to have an option to decide which process to kill based exclusively on VmRSS value

@rfjakob
Copy link
Owner

rfjakob commented Aug 18, 2019

In my case there's a lot of memory-eager applications

Hi! What kind of applications are we talking about here?

@AbsurdlySuspicious
Copy link
Author

AbsurdlySuspicious commented Aug 19, 2019

Well, in my case these are firefox, intellij idea, lutris (wine launcher), etc. Actually, whenever I open htop with sort by RES, I'm seeing a big wall of zeroes in OOM column, except for several rather small applications. I can figure out that this behavior isn't normal, but I really can't see for a solution other than, for example, forcefully set adj of all processes to zero by cron, which is surely somewhat clumsy approach.

@rfjakob
Copy link
Owner

rfjakob commented Aug 19, 2019 via email

@hakavlad
Copy link
Contributor

firefox, intellij idea, lutris

These applications do not set oom_score_adj=-1000.

@AbsurdlySuspicious Please run https://github.com/hakavlad/nohang/blob/master/oom-sort and print output here. This script prints also oom_score_adj values.

@AbsurdlySuspicious
Copy link
Author

@hakavlad
Copy link
Contributor

@AbsurdlySuspicious Seems like you set OOMScoreAdjust=-1000 in user.slice or in session.scope.

I'd like to see your sudo nohang -p output with extra_table_info = cgroup_v1.
(just clone nohang, change config and start sudo ./nohang -p, output like follow: http://okturing.com/src/6765/body)

@hakavlad
Copy link
Contributor

hakavlad commented Aug 20, 2019

@AbsurdlySuspicious Try also:

$ grep -sr "OOMScoreAdjust=" "/lib/systemd" "/etc/systemd" "/run/systemd"

It will display units with modified oom_score_adj.
Output like follow:

/lib/systemd/system/snapd.service:OOMScoreAdjust=-900
/lib/systemd/system/systemd-udevd.service:OOMScoreAdjust=-1000
/lib/systemd/system/dbus.service:OOMScoreAdjust=-900
/lib/systemd/system/dm-event.service:OOMScoreAdjust=-1000

@AbsurdlySuspicious
Copy link
Author

@hakavlad grep gives me *.service matches only (dbus, dm-event, etc). I'll try running nohang slightly later

@hakavlad
Copy link
Contributor

dbus, dm-event, etc

Please show all output.

@AbsurdlySuspicious
Copy link
Author

Binary file /lib/systemd/system-generators/systemd-cryptsetup-generator matches
Binary file /lib/systemd/systemd matches
/lib/systemd/system/postgresql.service:OOMScoreAdjust=-200
/lib/systemd/system/systemd-swap.service:OOMScoreAdjust=-1000
/lib/systemd/system/systemd-udevd.service:OOMScoreAdjust=-1000
/lib/systemd/system/ananicy.service:OOMScoreAdjust=-999
/lib/systemd/system/dbus.service:OOMScoreAdjust=-900
/lib/systemd/system/systemd-coredump@.service:OOMScoreAdjust=500
/lib/systemd/system/dm-event.service:OOMScoreAdjust=-1000
Binary file /lib/systemd/libsystemd-shared-242.so matches

Nothing related to slices or scopes, just like i said

@rfjakob
Copy link
Owner

rfjakob commented Aug 26, 2019

My guess: Ananicy does this, see https://github.com/Nefelim4ag/Ananicy/blob/6181b1dd3954c659eae905f0320c9c31562ecdb6/ananicy.py#L228 .
Please try disabling or uninstalling ananicy, reboot, and run oom-sort again.

@rfjakob
Copy link
Owner

rfjakob commented Aug 26, 2019

Also, looking at https://gist.github.com/AbsurdlySuspicious/2cc0988c5261a1ff595b4989c9a8d04e , ananicy is the first (lowest PID) process that has a -999 value for oom_score_adj. (other than systemd-udevd, but this has -1000 on my system as well)

@AbsurdlySuspicious
Copy link
Author

You're right, it was ananicy, I've disabled oom_score_adj applying and everything is fine now. Thank you guys for helping me with this.

I'd still like you to consider keeping this as low-prio feature request, as it still can be helpful in some cases, even if they're caused by mistake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants