Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oomd fails to see the problem if SwapTotal=0 and MemAvailable=0 #80

Closed
hakavlad opened this issue Aug 11, 2019 · 8 comments
Closed

oomd fails to see the problem if SwapTotal=0 and MemAvailable=0 #80

hakavlad opened this issue Aug 11, 2019 · 8 comments

Comments

@hakavlad
Copy link

hakavlad commented Aug 11, 2019

May be it should be documented.

Case:
SwapTotal=0
oomd started with -i 1

I ran memory hog, and at the end system hangs, oomd doesn't kill anything.

Maybe oomd should acts like earlyoom https://github.com/rfjakob/earlyoom and nohang https://github.com/hakavlad/nohang:
if MemAvailable < mem_threshold and SwapFree < swap_threshold, not only if SwapFree < threshold.

@hakavlad hakavlad changed the title oomd fails if SwapTotal=0 oomd fails to see the problem if SwapTotal=0 Aug 11, 2019
@hakavlad hakavlad changed the title oomd fails to see the problem if SwapTotal=0 oomd fails to see the problem if SwapTotal=0 and MemAvailable=0 Aug 11, 2019
@hakavlad
Copy link
Author

oomd doesn't help you in the case described in https://lkml.org/lkml/2019/8/4/15

@danobi
Copy link
Contributor

danobi commented Aug 12, 2019

PSI based oom killing doesn't work well without swap. Without swap, the system runs out of physical memory too fast and enter the "death spiral" before oomd has time to react.

With swap on, the system will get slowed down with swapping anonymous pages for long enough for oomd to react. In general, having swap on non-latency-critical systems is pretty good. Here's an excellent article by my colleague: https://chrisdown.name/2018/01/02/in-defence-of-swap.html .

@danobi
Copy link
Contributor

danobi commented Aug 12, 2019

Closing b/c this will be a wontfix. We can further discuss here if you want here.

@danobi danobi closed this as completed Aug 12, 2019
@danobi
Copy link
Contributor

danobi commented Aug 12, 2019

May be it should be documented.

Updating the docs now

@hakavlad
Copy link
Author

hakavlad commented Aug 16, 2019

PSI based oom killing doesn't work well without swap

Just don't use PSI-based oom killing without swap, use MemAvailable-based oom killing instead.

@hakavlad
Copy link
Author

hakavlad commented Aug 16, 2019

runs out of physical memory too fast and enter the "death spiral" before oomd has time to react

It shows the next problem: min interval is 1 sec.
For example, earlyoom has intereval 0.1 - 1 sec (and tiny CPU usage), it is enough to handle "runs out of physical memory too fast" and prevent system hang or OOM condition.

@hakavlad
Copy link
Author

hakavlad commented Aug 16, 2019

With swap on, the system will get slowed down

This is the main reason why many do not use swap. If you want to get a full-time responsive desktop, turn off the swap and use earlyoom (oomd is worst desktop solution regardless of swap availability).

@danobi
Copy link
Contributor

danobi commented Aug 16, 2019

PSI based oom killing doesn't work well without swap

Just don't use PSI-based oom killing without swap, use MemAvailable-based oom killing instead.

That might work for simple workloads, but IMO this is a local optimum. For example, how would this work with cgroups that reserve chunks of memory via memory.min? What if a host is configured with a high min_free_kbytes? What if the host runs out of MemAvailable memory but there's a lot of reclaimable memory in use (eg page cache)? Granted, these sorts of things are generally not a concern on desktop linux systems. Mostly server workloads.

If you really want, oomd can also support this. We can write a MemoryAvailable plugin. You don't have to use the pressure based plugins in your config.

This is the main reason why many do not use swap. If you want to get a full-time responsive desktop

I don't believe this is true, especially on modern systems with SSDs. This may have been true in the 90's and early 2000s. Swap these days usually leads to more equitable memory management. For example, dirty pages can always be flushed but anonymous memory is essentially mlock'd without swap. With swap the system can swap out cold anon pages and leave more room for hot pages in main memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants