Skip to content
This repository has been archived by the owner on Feb 4, 2020. It is now read-only.

ThreadPoolExecutor / concurrent.futures.as_completed prevent cProfile from providing actual info #364

Open
Artalus opened this issue Nov 15, 2019 · 0 comments

Comments

@Artalus
Copy link

Artalus commented Nov 15, 2019

Basically, what I get after combining pstats:

This doesn't help me if I want to profile processSingleSource() which is called inside scheduleJobs. What I actually want to see:

Notice how instead of scheduleJobs->as_completed->wait I get actual detalization of functions called inside processSingleSource.

I managed it by replacing ThreadPoolExecutor construct with simple processSingleSource() call, asserting that len(sourceFiles) == len(objectFiles) == 1. In my use case (CMake + Ninja) there is always 1 source file per 1 compiler call, so using thread pool becomes a waste of syscalls, since it will always create 1 thread to do the job and return immediately after this thread is done.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant