New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add profile page for event loop thread #2144

Merged
merged 20 commits into from Aug 19, 2018

Conversation

Projects
None yet
1 participant
@mrocklin
Member

mrocklin commented Jul 29, 2018

We have nice statistical profiling for the worker threads of each worker that is visible from the bokeh diagnostic pages. However it would be nice to also get this same kind of information about the main administrative thread within the scheduler and workers themselves. This will help us to understand performance costs generally and tune the scheduler and worker codes.

This PR copies and modifies the profile code to operate on each event loop and provides a new route (currently not advertised) on the dashboard. This helps us to understand the costs of things like asyncio and how they affect our general performance.

image

@mrocklin mrocklin merged commit dbb529b into dask:master Aug 19, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@mrocklin mrocklin deleted the mrocklin:profile-loop branch Aug 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment