Feature: monitor external python process, possibly by injecting a stub?

[hhimanshu]

Hello @zhuyifei1999,

I wanted to learn if I can use guppy to monitor an external process? My use case is that a python process is already running. I want to monitor the heap size over hours/days. It is a daemon, a long-running process, and based on how to use the guppy3, it seems I need to be inside that process to query the heap, this is not desirable for my use case.

Could you help to share your thoughts with me?
Thank you

[zhuyifei1999]

No, not at the moment.

Guppy uses Python's internal traversal, tp_traverse, to find what each object refers to (see: stdtypes.c#L359 hv.c#L298). This mechanism is also used by the python garbage collector. So at least, guppy needs a stub in the target process to figure out the reference tree. Maybe in the future guppy can use a debugger-like API like ptrace to perform the operation in any target PID, but given that there are so many types Python has, writing the "referees by object" function for each function type could be a huge amount of work. It can't just 'use' the tp_traverse without a way to simulate its execution....

However, if you have Python < 3.9, you can use remote monitor. It will spawn a subinterpreter in a new thread and run guppy from the new subinterpreter, with its stdio attached to a TCP connection. There you can use guppy without interrupting the normal process, though be careful that a lot of libraries don't work well with subinterpreters. Unfortunately, Python 3.9 completely breaks multithreaded subinterpreters (see commit 15fa02d), I'm considering under Python 3.9+ just spawn a new thread and have guppy overate from within, but without subinterpreters this loses the stdio redirection and I don't know of an easy way to workaround :(

[hhimanshu]

Thanks for your detailed response. I recently came across https://github.com/facebookincubator/memory-analyzer which does exactly that. I hope that this gives you some ideas as well

[zhuyifei1999]

Interesting. It seems to inject a piece of python code into the the target process via gdb. Not sure I agree with the runtime safety of doing so but it does sound like an interesting concept.