Repository navigation
Single flame graph output for multiprocessing? #951
|
Hi there, first of all thank you very much for this great tool! I'm trying to profile a python script that uses a ProcessPoolExecutor and the My setup:
Using I can create a flame graph for each subprocess, but there can be lots of them. It would be great if everything could be condensed in a single html, maybe with something like the split-threads view. Also: the capture files can be large, and memray suggests to use the |
Replies: 1 comment 2 replies
|
One report for all processes: no. memray flamegraph reads one capture file, so it is one report per child. Give each child its own -o, because the default report name drops the PID and every child gets the same name. The .bin.PID names are by design: a child writes to the parent's file name plus its PID. Empty files with --aggregate and --follow-fork: not your setup. The aggregated file is written only when tracking ends, and pool workers exit through os._exit, which skips that. I reproduced it with memray 1.20.0. For pool workers keep the default format. |
One report for all processes: no. memray flamegraph reads one capture file, so it is one report per child. Give each child its own -o, because the default report name drops the PID and every child gets the same name.
The .bin.PID names are by design: a child writes to the parent's file name plus its PID.
Empty files with --aggregate and --follow-fork: not your setup. The aggregated file is written only when tracking ends, and pool workers exit through os._exit, which skips that. I reproduced it with memray 1.20.0. For pool workers keep the default format.