Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add profile page for event loop thread #2144

Merged
merged 20 commits into from Aug 19, 2018
Merged

Add profile page for event loop thread #2144

merged 20 commits into from Aug 19, 2018

Conversation

@mrocklin
Copy link
Member

@mrocklin mrocklin commented Jul 29, 2018

We have nice statistical profiling for the worker threads of each worker that is visible from the bokeh diagnostic pages. However it would be nice to also get this same kind of information about the main administrative thread within the scheduler and workers themselves. This will help us to understand performance costs generally and tune the scheduler and worker codes.

This PR copies and modifies the profile code to operate on each event loop and provides a new route (currently not advertised) on the dashboard. This helps us to understand the costs of things like asyncio and how they affect our general performance.

image

@mrocklin mrocklin force-pushed the mrocklin:profile-loop branch from 6860afc to 9767c77 Jul 29, 2018
@mrocklin mrocklin mentioned this pull request Aug 2, 2018
7 of 7 tasks complete
@mrocklin mrocklin force-pushed the mrocklin:profile-loop branch from 9767c77 to 49650a3 Aug 2, 2018
@mrocklin mrocklin force-pushed the mrocklin:profile-loop branch 3 times, most recently from 6d055fc to 0906bc6 Aug 2, 2018
@mrocklin mrocklin force-pushed the mrocklin:profile-loop branch from f008518 to 2fbe005 Aug 17, 2018
mrocklin added 5 commits Aug 17, 2018
@mrocklin mrocklin merged commit dbb529b into dask:master Aug 19, 2018
2 checks passed
2 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@mrocklin mrocklin deleted the mrocklin:profile-loop branch Aug 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

1 participant