[macOS] cmdline() raises NoSuchProcess on EINVAL

[tamruta]

Recreated the problem I was facing:

process_hash = "login"  
for proc in psutil.process_iter():
    if process_hash in proc.name():
        server_process_running = True
        print(proc)
        print(str(proc.as_dict(['cmdline'])))  # this is where the problem rises 

psutil.Process(pid=3629, name='login', status='running', started='09:16:35')
Traceback (most recent call last):
  File "/opt/anaconda3/envs/trust/lib/python3.10/site-packages/psutil/_psosx.py", line 346, in wrapper
    return fun(self, *args, **kwargs)
  File "/opt/anaconda3/envs/trust/lib/python3.10/site-packages/psutil/_psosx.py", line 404, in cmdline
    return cext.proc_cmdline(self.pid)
ProcessLookupError: [Errno 3] assume no such process (originated from sysctl(KERN_PROCARGS2) -> EINVAL)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/test.py", line 14, in <module>
    print(str(proc.as_dict(['cmdline'])))
  File "/opt/anaconda3/envs/trust/lib/python3.10/site-packages/psutil/__init__.py", line 528, in as_dict
    ret = meth()
  File "/opt/anaconda3/envs/trust/lib/python3.10/site-packages/psutil/__init__.py", line 681, in cmdline
    return self._proc.cmdline()
  File "/opt/anaconda3/envs/trust/lib/python3.10/site-packages/psutil/_psosx.py", line 351, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: process no longer exists (pid=3629, name='login')

When I looked online about similar errors, some suggested it was caused by a zombie process. So I tried to kill the process and the terminal session closed.

I'm not sure if this is a bug or a feature, but this only happened when I updated psutil from 5.6.7 to 5.9.4

[giampaolo]

That's a race condition: between the for loop iteration and the proc.as_dict() call the process disappears. You have to ways to solve this problem. You either specify what process attributes you're interested in up front:

for proc in psutil.process_iter(attrs=["name", "cmdline"]):
      print(proc.info["name"])
      print(proc.info["cmdline"])

...or you catch the exception later:

for proc in psutil.process_iter():
    try:
        print(proc.name())
        print(proc.cmdline())
    except (psutil.NoSuchProcess, psutil.AccessDenied):
        continue

[tamruta]

Thank you! I used the first method

[tamruta]

@giampaolo I'm confused why the process no longer exists when the terminal session is still open. The race shouldn't really exist because the process never closed

And it seems to be a new problem that didn't happen in 5.6.7

[jacob-lee]

Perhaps this is expected behavior, however I find it surprising on two counts.

First, I have independently confirmed just now that no such exception is raised with version 5.6.7, but after upgrading to 5.9.4, the exception is thrown.

Secondly, it is the same process that consistently throws the error on my machine (M1 Macbook Pro, Ventura 13.0.1): the 'login -pf username' process running under root. Moreover, the process is still running under the same pid.

I suggest that this warrants re-opening the issue.

[giampaolo]

Mmm... even if the original python code had race condition, this may indeed be an actual bug. Error originates from here:

psutil/psutil/arch/osx/process_info.c

Line 135 in c6b7145

NoSuchProcess("sysctl(KERN_PROCARGS2) -> EINVAL");

Is the process a zombie? Can you guys please catch the NoSuchProcess exception and then print the process status()?

[tamruta]

The status is running:
console psutil.Process(pid=3629, name='login', status='running', started='09:16:35')

It seems to be caused by Mac's permissions to root processes. When the code is run on sudo it works without any errors. Also there is no error on linux or windows

[jacob-lee]

Hmm. That seems to be right. No exception raised if I run the python process with sudo.

[giampaolo]

Note to self: the problem is that EINVAL occurring on sysctl() should be translated into AccessDenied instead of NoSuchProcess, because the PID is still alive and is not a zombie. As such, raisingAccessDenied is really the best we can do in this circumstance (also confirmed by the fact that the problem does not occur as root user, as per @jacob-lee comment).

In order to fix this I'll have to refactor the C code a bit though, so this may take a while.