Skip to content

Commit

Permalink
[Test] Ignore psutil.AccessDenied when gathering per-process memory i…
Browse files Browse the repository at this point in the history
…nfo upon an OOM. (#16123)
  • Loading branch information
clarkzinzow committed May 27, 2021
1 parent 9c73591 commit cd71d5e
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions python/ray/_private/memory_monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,17 @@ def get_message(used_gb, total_gb, threshold):
# issue for more detail:
# https://github.com/ray-project/ray/issues/14929
continue
except psutil.AccessDenied:
# On MacOS, the proc_pidinfo call (used to get per-process
# memory info) fails with a permission denied error when used
# on a process that isn’t owned by the same user. For now, we
# drop the memory info of any such process, assuming that
# processes owned by other users (e.g. root) aren't Ray
# processes and will be of less interest when an OOM happens
# on a Ray node.
# See issue for more detail:
# https://github.com/ray-project/ray/issues/11845#issuecomment-849904019 # noqa: E501
continue
proc_str = "PID\tMEM\tCOMMAND"
for rss, pid, cmdline in sorted(proc_stats, reverse=True)[:10]:
proc_str += "\n{}\t{}GiB\t{}".format(
Expand Down

0 comments on commit cd71d5e

Please sign in to comment.