decoupled C library, without python

[justinmk]

The code in https://github.com/giampaolo/psutil/blob/master/psutil/arch/ is quite a trove of useful cross-platform voodoo. It's the most comprehensive that I've found. Projects like libuv are more conservative.

Has there been discussion about pulling the psutil C source into a separate project which could be included by other non-python C projects? Or, do you know of any efforts to do so? Or projects with similar goals?

How much effort would you estimate for this task? I would guess it's "just" a matter of removing the python bits.

I also wondered if https://github.com/micropython/micropython could be used as a bridge somehow, for projects that don't want to depend on the python universe, but I didn't check if that project is compatible with the python C API.

[giampaolo]

Yes, that is something I've been thinking about for a long time. One project I'm aware of which provides a "pure" C layer with on top the bindings for other languages is sigar.
psutil is getting to a point which is pretty much feature-complete. The next more logical step in terms of new functionality would be adding support for some exotic platform (e.g. cygwin) or start porting it to other languages (which is already happening BTW).
That's not a simple task though:

• cPython API is used extensively throughout all C code and the C code should return basic C types instead
• the implementation for platforms such as Linux is 80% Python. That would have to be rewritten from scratch in C.
• other platforms such as Windows also have a non-negligible amount of Python logic which would have to be rewritten in C. Also there is the main the main Python module.

How much effort would you estimate for this task?

A lot. This is massive and also has a cost in terms of maintainability ('cause C is less maintainable than Python) and stability ('cause you replace code which has been tested and used in production for years). I wouldn't go as far as saying it would mean starting from scratch but it's close. I imagine this as a 3-step process:

1. implement the pure C layer meaning getting rid of the cPython API and moving the Python logic to C. This may be a separate psutil-lib project or it may grow inside psutil itself.
2. add a Python wrapper on top of it
3. gradually add wrappers for other languages

It's hard to tell how long this would take but it's surely in the order of months for steps 1) and 2), assuming a full-time investment (don't forget psutil runs on many platforms). This of course does not include 3), meaning other languages' wrapper implementation, unit tests, documentation, rethinking/adjustment of APIs and a proper and automated packaging/distribution, which implies a deep familiarity with them. In that case we may as well reach and exceed 1 year of development.

[justinmk]

Thanks for the detailed answer!

implement the pure C layer meaning getting rid of the cPython API and moving the Python logic to

Ok, I didn't realize that python was used in the arch/ layer for more than passing around data.

I agree a big refactoring sounds horrible. Using https://github.com/micropython/micropython could be a very interesting hack, though maybe a different type of unpleasant (build-system) work.