Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OOM Killer kills Scylla / Scylla-JMX - validate whats running / memory setup #1242

Closed
slivne opened this issue Jan 15, 2021 · 0 comments · Fixed by #1249
Closed

OOM Killer kills Scylla / Scylla-JMX - validate whats running / memory setup #1242

slivne opened this issue Jan 15, 2021 · 0 comments · Fixed by #1249

Comments

@slivne
Copy link

slivne commented Jan 15, 2021

Please make sure that this is Advisor related.

Describe the issue logically
OOM Killer - kills Scylla / Scylla-JMX - this happens under memory pressure and is extreme case - need to validate there are no process taking a lot of memory on the system

Its also an indication that swap may not be defined as Scylla-JMX can be moved to swap

Describe the condition when the issue happens
oom-killer logs

Jan 15 03:08:35 ip-xxxxxx kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Jan 15 03:08:35 ip-xxxxxx kernel: [    631]     0   631    12171      527   126976        0         -1000 systemd-udevd
Jan 15 03:08:35 ip-xxxxxx kernel: [    685]     0   685    13887      111   135168        0         -1000 auditd
Jan 15 03:08:35 ip-xxxxxx kernel: [    789]    81   789    16582      148   147456        0          -900 dbus-daemon
Jan 15 03:08:35 ip-xxxxxx kernel: [   2666]     0  2666    28236      258   262144        0         -1000 sshd
Jan 15 03:08:35 ip-xxxxxx kernel: [   2977]   996  2977 4295083049 22677315 182317056        0          -950 scylla
Jan 15 03:08:35 ip-xxxxxx kernel: [  14856]     0 14856     2173        0    61440        0             0 systemd-cgroups
Jan 15 03:08:35 ip-xxxxxx kernel: [  14857]     0 14857     2140        0    57344        0             0 systemd-cgroups
Jan 15 03:08:35 ip-xxxxxx kernel: [  14865]     0 14865       79        0    32768        0             0 systemd-cgroups
Jan 15 03:08:35 ip-xxxxxx kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=scylla,pid=2977,uid=996
Jan 15 03:08:35 ip-xxxxxx kernel: Out of memory: Killed process 2977 (scylla) total-vm:17180332196kB, anon-rss:90709260kB, file-rss:0kB, shmem-rss:0kB, UID:996 pgtables:178044kB oom_score_…

What the user should do when this happens
validate swap is define and there are no rogue process taking a lot of memory

Any Other info or Links.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants