Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError: [WinError 0] from QueryFullProcessImageNameW #1662

Closed
amerry opened this issue Jan 14, 2020 · 12 comments · Fixed by #1677
Closed

OSError: [WinError 0] from QueryFullProcessImageNameW #1662

amerry opened this issue Jan 14, 2020 · 12 comments · Fixed by #1677

Comments

@amerry
Copy link

amerry commented Jan 14, 2020

I occasionally see this with psutil 5.6.7:

File "...\lib\site-packages\psutil\__init__.py", line 731, in name
  name = self._proc.name()
File "...\lib\site-packages\psutil\_pswindows.py", line 709, in wrapper
  raise convert_oserror(err, pid=self.pid, name=self._name)
File "...\lib\site-packages\psutil\_pswindows.py", line 699, in convert_oserror
  raise exc
File "...\lib\site-packages\psutil\_pswindows.py", line 707, in wrapper
  return fun(self, *args, **kwargs)
File "...\lib\site-packages\psutil\_pswindows.py", line 782, in name
  return py2_strencode(os.path.basename(self.exe()))
File "...\lib\site-packages\psutil\_pswindows.py", line 709, in wrapper
  raise convert_oserror(err, pid=self.pid, name=self._name)
File "...\lib\site-packages\psutil\_pswindows.py", line 699, in convert_oserror
  raise exc
File "...\lib\site-packages\psutil\_pswindows.py", line 707, in wrapper
  return fun(self, *args, **kwargs)
File "...\lib\site-packages\psutil\_pswindows.py", line 791, in exe
   exe = cext.proc_exe(self.pid)
OSError: [WinError 0] The operation completed successfully: '(originated from QueryFullProcessImageNameW)'

I haven't figured out how to reproduce it (I see it happening in scripts, but whatever process caused it has gone by the time I try to get more info out). It looks like QueryFullProcessImageNameW isn't always setting error info properly on failure, though, which then means the error isn't being translated to a psutil error.

I'm not sure there's anything you can reasonable do about this, but I thought you might like to be aware of it.

@giampaolo
Copy link
Owner

Oh! This occurred before: #1394. And it was fixed in 6f4a622 but somehow I removed that change.

@giampaolo
Copy link
Owner

giampaolo commented Jan 14, 2020

It turns out the fix was about Windows XP [1], so this is a new problem.
[1] https://github.com/giampaolo/psutil/blob/release-5.6.7/psutil/_psutil_windows.c#L651-L654
This looks like a bug (or misdesign) with the Windows API:
https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-queryfullprocessimagenamew
...which doesn't do as promised. Quote:

If the function succeeds, the return value is nonzero.
If the function fails, the return value is zero. To get extended error information, call GetLastError.

Fixed in efaa9e0.

@otrejoso
Copy link

Do you guys know when the new version having this fix will be coming out? 🤔

@giampaolo
Copy link
Owner

make win-download-wheels will get you latest (unreleased) wheels

@otrejoso
Copy link

Thanks for the command @giampaolo, but I just installed the new 5.7.0 psutil wheel and I am still having the same error 😞

C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\testdriver\executor.py:218: in run_command
    LocalExecutor.execute(executable, timeout=timeout)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\testdriver\executor.py:190: in execute
    NoLogLocalExecutor.execute(executable, timeout)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\testdriver\performance_functions.py:23: in wrapper
    value = func(*args, **kwargs)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\testdriver\executor.py:84: in execute
    proc_name, mem, local_children = find_memory_usage(proc)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\testdriver\performance_functions.py:74: in find_memory_usage
    children.add((c.pid, c.name()))
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\__init__.py:630: in name
    name = self._proc.name()
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:680: in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:670: in convert_oserror
    raise exc
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:678: in wrapper
    return fun(self, *args, **kwargs)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:753: in name
    return py2_strencode(os.path.basename(self.exe()))
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:680: in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:670: in convert_oserror
    raise exc
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:678: in wrapper
    return fun(self, *args, **kwargs)
C:\_\k0\project\test\tests_pti\.pti_artifactory\venv\lib\site-packages\psutil\_pswindows.py:759: in exe
    exe = cext.proc_exe(self.pid)
E   OSError: [WinError 0] The operation completed successfully: '(originated from QueryFullProcessImageNameW)'
----------------------------------------------------- Captured stdout setup -----------------------------------------------------

@giampaolo
Copy link
Owner

Mmm I don't see how that can happen. I suspect that you're still using the old installation. Try make uninstall multiple times first.

@otrejoso
Copy link

otrejoso commented Jan 30, 2020

I am working in a fresh virtual environment and sort of the trace points at _pswindows.py:759 which matches whatever is on master. 🤔 ... I don't see how I can be using an old installation unless I am missing something.

@otrejoso
Copy link

One more update, I started to go back psutil versions and installing 5.5.1 works fine. I don't see any error. The moment I went to 5.6.0, I started to get the

   OSError: [WinError 0] The operation completed successfully: '(originated from QueryFullProcessImageNameW)'

Not sure if this helps to narrow things down, but I think I can workaround it like this in the meantime.
Let me know if you want me to try other things. 😄

@giampaolo
Copy link
Owner

I am working in a fresh virtual environment and sort of the trace points at _pswindows.py:759 which matches whatever is on master. thinking ... I don't see how I can be using an old installation unless I am missing something.

Sometimes the compiled pyd file from a previous installation sticks around. That's why I added make uninstall. Anyway, I am not sure this is the problem. I suspect it may be because the fix to this problem is supposed to be here:

if (GetLastError() == 0)
AccessDenied("QueryFullProcessImageNameW (forced EPERM)");
else
PyErr_SetFromOSErrnoWithSyscall("QueryFullProcessImageNameW");

The best thing you can do is install Visual Studio, put some debugging printf()s in the C code and see what happens:
https://github.com/giampaolo/psutil/blob/master/INSTALL.rst#windows

@otrejoso
Copy link

otrejoso commented Jan 31, 2020

So I ended up modifying those lines as follows

static PyObject *
psutil_proc_exe(PyObject *self, PyObject *args) {
    DWORD pid;
    HANDLE hProcess;
    wchar_t exe[MAX_PATH];
    unsigned int size = sizeof(exe);

    if (! PyArg_ParseTuple(args, _Py_PARSE_PID, &pid))
        return NULL;

    hProcess = psutil_handle_from_pid(pid, PROCESS_QUERY_LIMITED_INFORMATION);
    if (NULL == hProcess)
        return NULL;

    memset(exe, 0, MAX_PATH);
    if (QueryFullProcessImageNameW(hProcess, 0, exe, &size) == 0) {
        // https://github.com/giampaolo/psutil/issues/1662
        DWORD err_ret = GetLastError();
        int bufferSize = WideCharToMultiByte(CP_UTF8, 0, exe, -1, NULL, 0, NULL, NULL);
        char *utfString = (char*)malloc(bufferSize);
        WideCharToMultiByte(CP_UTF8, 0, exe, -1, utfString, bufferSize, NULL, NULL);
        if (err_ret == 0)
            AccessDenied("QueryFullProcessImageNameW (forced EPERM)");
        else
            PyErr_MySetFromOSErrnoWithSyscall("QueryFullProcessImageNameW", err_ret, utfString, pid);
        CloseHandle(hProcess);
        return NULL;
    }
    CloseHandle(hProcess);
    return PyUnicode_FromWideChar(exe, wcslen(exe));
}

Where the code is printing in PyErr_MySetFromOSErrnoWithSyscall("QueryFullProcessImageNameW", err_ret, utfString, pid);

Looking at the new error, I have the following message:
E OSError: [WinError 0] The operation completed successfully: '(originated from QueryFullProcessImageNameW, 31, , PID: 3332)'

err_ret is 31, and based on https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-, 31 is:

ERROR_GEN_FAILURE
    31 (0x1F)
    A device attached to the system is not functioning.

Do you know what that means? 🤔 Am I looking at the right place?

As an additional step, I enabled the Audit Process Tracking Security Policy to figure out what was the process that it was complaining about. It is our own program that we are starting, but based on Event Viewer. Our program is exiting correctly and gracefully:

Process Information:
	Process ID:	0xd04  <--- NOTE HERE HEX WHICH MATCHES MY PID ABOVE
	Process Name:	C:\_\k0\project\deploy_project\bin64\project.exe
	Exit Status:	0x0

@giampaolo
Copy link
Owner

#1677 has implicitly fixed this issue.

@otrejoso
Copy link

otrejoso commented Feb 3, 2020

#1677 indeed fixes our issue. We no longer see the error. Thanks @giampaolo 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants