New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prototype for Cygwin support module (not ready to merge) #998
Commits on Mar 14, 2017
-
-
Initial addition of a cygwin platform module.
Some common utilitiy functions were moved out of _pslinux into _common. Most of cygwin's Process implementation is directly copied from the linux Proces, minus a few very Linux-specific parts. I hope to eliminate the code duplication later with further refactoring. Most tests fail due to some key Process methods not having implementations yet (mostly because their implementations for Linux do not work on Cygwin).
-
First pass at implementing Process.create_time for Cygwin
I left this unimplemented originally just because I wanted to take the time to understand what I was doing before continuing. Turns out it can be implemented almost the same as on Linux, except that the process start time is in the 22nd field of /proc/<pid>/stat instead of the 21st. I can't find enough documentation for Cygwin's /proc filesystem, but I confirmed this in the source. Other than that the semantics are the same as on Linux. So much of this was copied agained from the Linux module, but I will refactor later.
-
Added initial virtual_memory() and swap_memory() implementations.
These are much simpler than on Linux, and effectively the same as the existing Windows implementations of these functions, though easier to implement on Cygwin via /proc/meminfo
-
Add previously missing pfullmem namedtuple, which some of the tests
expect to find. Currently it is identical to pmem; will return later to seeing what additional memory info we can provide on Cygwin
-
Fix broken Process.uids and Process.gids on Cygwin
Later this should be able to share its implementation with the Linux module but we'll come back around to that.
-
Add cpu_count_logical(), cpu_count_physical(), and cpu_stats() for
Cygwin Copied almost verbatim from the Linux module.
Commits on Mar 15, 2017
-
Compile at least the arch/windows/process_info.c module into the
cygwin module so that the functions in it can be reused there. This required some tweaks to ntextapi.h, since a few of the types defined there are already included in the winimpl.h included with Cygwin. This includes are own implementation of PyErr_SetFromWindowsErr which is not defined in Python for Cygwin. TODO: Because this PyErr_SetFromWindowsErr is lifted, only slightly modified, from the Python source it should include a copyright notice to the effect. This implementation probably does not work quite right on Python 2 either--it should return an actual WindowsError there.
-
Add proc_cpu_affinity_get/set. This is copied exactly from the Windows
module for now. Will refactor later.
-
Initial disk_partitions() implementation for cygwin module.
Copied exactly from the Linux module and seems to work; will refactor later.
-
Initial implementation of disk_usage() for cygwin.
Using the posix implementation directly seems to just work.
-
Initial implementation of net_if_addrs() for the cygwin module.
The only obvious issue currently is that rather than the human-friendly interface names returned by this function in the normal Windows implementation, the interface names are instead some UUID looking things for each interface (the same as appear in the registry in various places to represent the interface). This is fine for now, but not ideal. Should be able to map these to friendlier names.
-
-
Move the py2_strencode utility from the windows module into common,
renaming it to 'encode', for symmetry with the 'decode' utility. This utility may also be useful in the Cygwin module, and is otherwise not Windows-specific, having more to do with Py2/3 compatibility.
-
Very rough first pass at implementing net_if_stats for Cygwin
This is mostly copied from the Windows module with a few small tweaks, but compiling this is a bit of a pain and somewhat fragile. Will have to come back to this for a closer look later, but for now I just want to get all of the API implemented as best I can.
-
Initial implementation of net_io_counters for the cygwin module.
As with net_if_stats, this is a bit of a mess right now to compile, and copies from the Windows module, but will clean up later.
-
Initial implementation of the net_connections() function for cygwin
Once again, mostly copied from the Windows module, with much refactoring needed.
-
Added a simple wrapper for cygwin_conv_path() from Cygwin's (very lim…
…ited) API. This will allow methods that return paths to return Cygwin paths instead of native Windows paths. I have yet to go through and determine all the places this should be used, but there are probably at least a few.
-
Add conversion from Cygwin PID to native Windows PID.
Whenever calling a C extension method that calls out to the Windows API make sure to use the Windows PID. This should fix a lot of issues where the wrong PID was being used.
-
Initial implementation of Process.memory_maps for the cygwin module.
Most of this is copied from the windows module with a few minor tweaks, necessitating later refactoring. This is also the first method to make proper use of the new winpath_to_cygpath function.
-
Fix cygwin's Process.cpu_affinity_get/set by copying the Python-level…
… implementations from the windows module (rather than the linux module) but using the winpid.
-
Initial implementation of disk_io_counters for Cygwin; copied almost …
…verbatim from the windows module.
-
Initial implementation of the psutil.users() function for the cygwin …
…module. This is copied mostly from the Linux module, though it needs to use the utmpx APIs. This seems to 'work' insofar as the API is available on cygwin, but it's not clear to me that the information returned is fully correct or useful. Will return to this if needed.
-
On cygwin (as on windows) Process.connections is just a wrapper aroun…
…d net_connections.
-
Added initial support for Process.environ, again copied from the wind…
…ows module (The linux module's method of looking in /proc doesn't work on cygwin's more limited /proc namespace)
-
Initial cygwin support for Process.io_counters and Process.num_ctx_sw…
…itches, mostly supported by the proc_info function copied from the windows module.
-
Add cygwin interface for Process.num_threads
(This is just a thin wrapper around proc_info())
-
-
Initial cygwin support for Process.threads
Copy/pasted from windows module, so this carries the usual admonition about refactoring.
-
The child process search that occurs if ppid_map is not available sho…
…uld fail gracefully on AccessDenied errors. If we get an AccessDenied trying to get the ppid of a process, it's *probably* not a child process of the process in question anyways.
-
Use the Process.memory_info implementation from the windows module
instead of going through Cygwin's /proc which turns out to be a bit buggy and for this purpose. This allows us to expose more information anyways such as USS in memory_info_full, just unfortunate that the Cygwin interface can't be relied on here.
-
Turns out there does not (for now) appear to be any straightforward way
to get the process environment on Cygwin. So we remove the functionality for now. See https://cygwin.com/ml/cygwin/2016-11/msg00205.html
-
After researching the subject extensively, I've determined it's not p…
…ossible to enumerate UNIX sockets on Cygwin Cygwin emulates UNIX sockets as a INET socket. Although it does list them in /proc/<pid>/fd as some 'socket:[N]' like on Linux, the 'N' here is only of internal meaning to Cygwin (this is true on Linux as well), but there is nothing resembling /proc/net/unix or anything else that allows one to map that number N to an actual socket file (such a file *does* exist, and in theory we could find it be enumerating ALL files on the system until it is found, but this is not practical). I can't find any other obvious way to do that. The only practical solution would be if Cygwin had an equivalent to /proc/net/unix.
-
Add Cygwin version info constants--these are not used anywhere yet bu…
…t may prove useful later
-
allow process_iter to skip processes on AccessDenied, which can happe…
…n when trying to query processes for ppid that one does not have access to (TODO: it may be possible to get the ppid by some other means in this case though)
-
-
Adds a new ps() interface for the posix tests that better clarifies the
platform-specific differences between how ps is called. Also implements significant support for Cygwin, where the built-in ps lacks many of the features of ps on other systems. This fixes a number of test errors on Cygwin (it also exposes a few new failures but those will be addressed separately).
-
Cygwin does not necessarily create utmp entries when you start a logi…
…n terminal, so 'who' can return empty--this fix allows the test to work even when there are no utmp entries.
-
Fix a corner case that can occur when a getmntent() entry's mount point
does not actually exist. This happened to be affecting my test runs.
-
Turns out the process creation time from /proc/<pid>/stat cannot be
relied on in Cygwin, due to a rounding error there which can cause it to return fluctuating results over time. This is really bad because process creation time is part of the 'identity' psutil util uses for a process when comparing it for equality to another process. This once again copies code from the windows module into the cygwin module, which I intend to refactor later. I also plan to address the upstream bug in Cygwin, but until that's fixed this workaround is needed.
-
-
It seems this test is having the same problem on Cygwin as on plain
Windows with processes not being truly destroyed quickly enough.
-
Make one small Cygwin-specific ifdef in psutil_net_if_addrs to suppor…
…t getting the 'friendly name' as opposed to the fixed UUID-based name of an interface. This is more consistent with the other network APIs on Windows/Cygwin which return the interface friendly names
-
Change net_if_stats() to return the same network interfaces as net_if…
…_addrs()--kind of a hack for now.
-
Make this test pass more reliably by allowing the ps output to includ…
…e one extra process for ps itself. (I'm not sure how this test worked on Linux since it normally includes ps itself too...)
-
As written, this test could not work when usernames contain spaces (w…
…hich is not uncommon on Cygwin)
-
Prevent this test from failing on Cygwin on processes running at a hi…
…gher privilege than the test runner
-
These tests are a mess on Python 2 due to inconsitent handling of non…
…-unicode strings
-
Skip this test on Cygwin, as there is no reliable way currently to ma…
…p from net_connections() results to file descriptors
-
Improved handling for a buggy (?) state that can occur on Cygwin wher…
…e a process is in the zombie state but its /proc/<pid>/stat file can no longer be read. See https://www.cygwin.com/ml/cygwin/2017-02/msg00187.html
-
Return fewer interfaces from net_io_counters on Cygwin, for parity wi…
…th the functionality in the Windows interface. This allows the test_nic_names test from the Windows tests to be reused and to pass on Cygwin.
-
More fixes for this test to work on Cygwin.
I'm worried it still might be a bit flakey though.
-
-
-
some of the exe tests pass most reliably on Cygwin if we always drop …
…the .exe right here
-
-
Add support for sensors_battery on Cygwin
(Copied directly from the Windows module for now)
Commits on Mar 23, 2017
-
Skip these tests on AppVeyor (under Cygwin only)
They are failing persistently due to fork errors. No matter how I rebase the affected DLLs it seems to have problems on these. I have given up trying to find out why for now.
-
-
-
More fork errors on Cygwin, now immediately upon running setup.py bui…
…ld (this did not happen before). Try a rebaseall immediately before setup.py build.
-
(previously this would mistakenly convert something like 'python2.7' to just 'python2'--if the latter is a symlink this can break things)
-
It seems that random fork errors can occur here on Cygwin, but not *always*. Retrying usually works.
-
Disable testing Python 3 on Cygwin in AppVeyor for now until I can spend
more time figuring out what the problem is.
Commits on Apr 11, 2017
Commits on Apr 26, 2017
-
Better way to get the major/minor version of Windows, without relying…
… on running a subprocess or cmd.exe. Doesn't return the build number unfortunately, but we're not using that right now anyways.
Commits on Aug 29, 2017
Commits on Nov 6, 2017
-
-
A much cleaner, better functioning, and more Cygwin-ish approach to e…
…rror handling from Python wrappers for Windows APIs. Instead of attempting to reimplement PyErr_SetFromWindowsError, on Cygwin we just replace those calls with a macro wrapping PyErr_SetFromErrno which uses the cygwin_internal API to map Windows error codes to POSIX error codes just in the same way Cygwin would. This appears to work correctly (for example trying to get info on a non-existent process returns errno ESRCH (No such process).
-
As noted in the comment, cygpid_to_winpid and winpid_to_cygpid can have
their pure Python implementations replaced with more efficient (and less racy) C implementations.
-
First move of a utility common to both the windows and cygwin modules.
Although I considered adding a new module for this purpose, for now it suffices to use the _common module for this (albeit a bit hackishly). In this case the C wrapper for this function makes sense to put in arch/windows.