Twisted asynchronous api for psutil.

[giampaolo]

From justin.v...@gmail.com on August 27, 2012 05:49:37

Proposal  
Twisted interface for psutil methods, psutil.Process, and psutil.Popen. 

On what platforms would this be available?  
All platforms that twisted and psutils both work on. 

Proposed API  
Provide asynchronous interfaces that would be safe for use with the twisted reactor. 

Are there existent implementations we can use as an example?  
This implementation works on my linux test machines. 
https://github.com/JustinVenus/twisted-psutil/blob/master/process.py 

Please provide any additional information below.  
If the psutil project isn't interested in merging this can I get signoff to try 
to send it to the twisted project for integration.  I ask for signoff because 
the process.test method is directly based on psutil.test.

Original issue: http://code.google.com/p/psutil/issues/detail?id=320

[giampaolo]

From g.rodola on August 27, 2012 03:32:27

IMO, except for maybe Process.get_children(), all other process methods should 
be quick enough to fit into any async loop "time window" without introducing 
problematic slowdowns, therefore I see no need to use threads in the first place.
Try this:

import time, os, psutil
t = time.time()
psutil.Process(os.getpid()).as_dict()
print time.time() - t

On my Linux box the code above took 0.0037 secs to complete.
If I add process children I get 0.0181 secs.
I expect that both 0.0037 and 0.0181 secs fit well in any async model.

Note that:
- as_dict() collects *all* process info, except its children.
- psutil's Linux implementation is already particularly slower compared to 
other platform implementations. On other platforms the code above is likely to 
run faster.

[giampaolo]

From justin.v...@gmail.com on August 27, 2012 08:51:33

Thank you for your feedback.  Linux is my top use case at the moment.  I
have observed that the open call followed by read pauses the reactor,
albeit by a small amount for single calls, however delays can quickly back
up the reactor and impact time sensitive calls.  I have an application
under development that needs to read a lot of the information that
psutil.Process exposes and then further process it (callback chains would
be very effective for this additional processing).  Unfortunately I have
run into significant reactor blocking due to all the reads and post
processing that I need to perform with this application.  This wrapper has,
in my case, alleviated the problem of blocking the reactor and thus the
application I am working on performs acceptably.  I am only opening the
feature request as it may be useful for someone else using twisted with
psutil.   As stated in my enhancement request, if the psutil project is ok
with the fact that I copied the test method from psutil.__init__ and the
psutil project is not interested in including this code in its source tree,
then I would like to send this code to the twisted project (MIT Licensed)
for possible inclusion in twisted.

[giampaolo]

From g.rodola on August 27, 2012 09:49:51

Hi Justin,
I don't know your specific use case but spawning a thread for *every* operation 
you do against the Process instance looks like a pretty inefficient approach to me.

Assuming speed is actually a problem I would use a *single* thread to collect 
all the process(es) info you need and use the twisted APIs to obtain/exchange 
the return value(s) you need from that Process object (hint: consider using 
Process.as_dict()).

I have nothing against you including parts of psutil source code as a patch for 
another project (Twisted) so feel free to do it.

As for now I'm not interested in including your patch as-is but I'd like to 
know what Twisted folks think about this, so in case you file an issue on their 
tracker please let me know.

Closing out as rejected for now.

Status: WontFix
Labels: -Priority-Low Priority-Medium

[giampaolo]

From justin.v...@gmail.com on August 27, 2012 10:03:08

Thank you.

Justin Venus