Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Cygwin port #1884

Closed
wants to merge 11 commits into from
Closed

WIP: Cygwin port #1884

wants to merge 11 commits into from

Conversation

embray
Copy link
Contributor

@embray embray commented Nov 19, 2020 •
edited

Adds Cygwin support to psutil, to resolve #82, #1755, #1856 and others.

Marked WIP as there are still several interfaces not implemented yet:

  • net_connections (partially done)
    • support AF_INET and AF_INET6 connections
    • support AF_UNIX connections
  • net_io_counters
  • net_if_stats
  • disk_usage
  • disk_io_counters
  • disk_partitions
  • users
  • Process.terminal
  • Process.num_ctx_switches
  • Process.num_threads
  • Process.open_files
  • Process.connections (done other than AF_UNIX connections which are handled by net_connections)
  • Process.num_fds
  • documentation updates

For those interfaces that can't be implemented directly through Cygwin, I'm working on incorporating piecemeal bits of the Windows module into the Cygwin module and performing conversions (e.g. of PIDs) where necessary.

Also includes a rudimentary Cygwin build in Appveyor.

@giampaolo I hope we can leave the past in the past and work together to move forward on this. I'm happy to incorporate any changes you request without argument.

Erik M. Bray added 11 commits November 19, 2020 16:26
@embray
Windows: Ignore dlls from running build_ext --inplace on Windows
e50d025
@embray
Cygwin: Initial Cygwin support module with just a bit more than bare-…
f57c690 
…minimum

functionality.

Many tests are set to skip on Cygwin for now, but can be re-enabled as
more functionality is added back in.

The C extension module is just a stub for now and might end up being
removed altogether, especially if we can add
[PyCygwin](https://pycygwin.readthedocs.io/en/latest/) as a dependency.
@embray
Cygwin: Add a simple Appveyor configuration for Cygwin.
d6184b7 
Squashed commit of the following:

commit d2305c5
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Thu Nov 19 12:29:57 2020 +0100

    Try building in a wheel in after_test

    Setting `artifacts: []` seems to have been pointless as well, but that's
    ok.  If we're building wheels it will be useful to have them uploaded as
    artifacts.

commit 21dbf27
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 21:26:16 2020 +0100

    It didn't like an empty list either, at least for after_test

    It just ignored it and tried to run the default after_test

commit f8776e3
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 21:21:24 2020 +0100

    It didn't like "off" for these, so try an empty list.

commit eb6454d
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 20:26:41 2020 +0100

    Skip this test on CI; no matter what I try the Python process only as
    PPID=1

commit 3e35641
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 20:04:39 2020 +0100

    Fix this test to hopefully work better on Cygwin

    It worked for me locally but that might have just been luck; on Appveyor
    it failed.

commit e1f7779
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:51:29 2020 +0100

    Disable this portion of print_sysinfo on Cygwin for now

commit 5082fe3
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:44:13 2020 +0100

    More skipIf conditions that currently crash on Cygwin

commit 66873c8
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:37:11 2020 +0100

    The condition in this skipIf crashes on Cygwin since users() is not
    implemented yet on Cygwin

commit bbea98a
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:31:39 2020 +0100

    Still need to explicitly install binutils, because otherwise Cygwin is
    selecting a too-old version.

    Maybe using apt-cyg to install packages would be better, but let's see
    if we really need that...

commit f8aabd9
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:27:41 2020 +0100

    Don't try to pass a bash script on the command line from within a
    windows cmd.exe shell

    I should have known better than to even try that.

commit 6ff678c
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 19:11:35 2020 +0100

    In practice, installing the full dev-env is proving a bit troublesome.

    It takes a long time; in particular to build grpcio which is broken on
    Cygwin anyways, but is listed as a dependency of one of the
    dependencies.

    I don't think we need all this just to build psutil and run te tests, so
    here's a simplified script based on the one for freebsd.

commit db4582a
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 18:49:15 2020 +0100

    Manually selecting binutils shouldn't be necessary, but just on a hunch
    that it fixes cffi compilation bug...

commit a30862f
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 18:25:35 2020 +0100

    * add ~/.local/bin to the path since the Makefile likes to pip install
      --user except when in a virtualenv

    * install libffi-devel for cffi

    * pass the correct PYTHON to make

commit 1fba10b
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 16:18:13 2020 +0100

    Try adding a minimal installation stage

commit 85a26f5
Author: E. Madison Bray <erik.bray@lri.fr>
Date:   Wed Nov 18 12:19:48 2020 +0100

    Test adding Cygwin-specific config to the appveyor.yml.

    It is not extraordinarily well-documented how the for: clause in
    appveyor.yml works, so for starters we're just seeing if adding a
    Cygwin-specific job works; i.e. does not run the default scripts.
@embray
Cygwin: add swap_memory by wrapping the Linux implementation
e96ed20
@embray
Cygwin: add net_if_addrs support
f9d3bb5 
Using the Linux implementation, which in turn is just the POSIX
implementation, seems sufficient.
@embray
Windows: fix segfault in psutil_net_connections when passed invalid
b3fcff5 
parameters
@embray
Cygwin: Add Process.winpid property for getting the Cygwin process's
6b5c398 
underlying Windows PID
@embray
Cygwin: add cygpid_to_winpid and winpid_to_cygpid utility functions
09f1a4e
@embray
Cygwin: Add the minimum bits necessary to be able to import
6dec7e9 
_psutil_cygwin as a replacement for _psutil_windows in _pswindows

This is admittedly a bit hacky--the goal here is to be able to import
_pswindows on Cygwin, but only use the bare minimum functionality from
it required for Cygwin.  The full functionality of _pswindows is not
needed, and might be onerous to try to compile.
@embray
Cygwin: Add _psutil_cygwin.net_connections by compiling in the
860b4ee 
arch/windows/socks.c module which provides the Windows implementation

Also requires arch/windows/process_utils.c and a few minor changes in
order for it to compile on Cygwin, including an implementation of
PyErr_SetFromWindowsErr for Cygwin.
@embray
Cygwin: partial support for net_connections and Process.connections
c23c136 
Works except for listing AF_UNIX sockets connections.  Since it's using
the Windows implementation of proc_net_connections, it naturally has no
concept of UNIX sockets.

It may be possible to add support for this with a Cygwin-specific
auxiliary function which would depend on understanding how Cygwin
implements UNIX sockets internally, and then digging around with some NT
APIs to find them.
@giampaolo
Copy link
Owner

giampaolo commented Nov 19, 2020 •
edited

@embray I'm sorry but being this a largely incomplete WIP (basically the same as the last one), I know it would take months of back & forth communication to decide where/how to move stuff, incorporate changes, etc. I could do that back in the days because I had much more free time, but not anymore. Incidentally, and as I said elsewhere, I'm planning to release Linux/OSX wheels, and then take a break, because psutil became too demanding for me alone in terms of time. Given the unusual dual nature of Cygwin which we both know (most stuff is already there, but it requires a lot of refactoring and moving stuff around), I came to the conclusion that I prefer to do this myself, if/when I'll have time. You can of course feel free to do this work in your fork, and if you reach a good point, feel free to point/mention it in the Cygwin ticket so that people can use it if they want.

@giampaolo giampaolo closed this Nov 19, 2020
@giampaolo
Copy link
Owner

giampaolo commented Nov 24, 2020 •
edited

embray@5d132f2, 2 hours ago

I leave this up to @giampaolo

You are now blocked from this repo, which I already put under full moderation [1] 5 days ago after I closed this PR, precisely because I was expecting something like this. Any further attempt to try to disrupt this project [2], for which I also receive some royalties, will not be tolerated

[1] people who didn't previously commit, unlike you, can't currently open tickets, comment or submit PRs
[2] #1483, #1342, deleted comment in which you insulted me, you keep updating this ticket for the past 4 days meaning you have no intention to respect my requests once again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
cygwin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Cygwin Support
2 participants
@embray @giampaolo