Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DON'T MERGE] Disable GC on scheduler at benchmark start #220

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

randerzander
Copy link
Contributor

@randerzander randerzander commented Jul 12, 2021

Based on, dask/distributed#4987, ran SF1k w/ GC disabled on the scheduler process, and observed total benchmark runtime improvement of about 5s. This was consistent across many runs.

Results from larger scale tests are pending.

Based on, dask/distributed#4987, ran SF1k w/ GC disabled on the scheduler process, and observed total benchmark runtime improvement of about 5s. This was consistent across many runs.
@randerzander randerzander changed the title Disable GC on scheduler at benchmark start [DON'T MERGE] Disable GC on scheduler at benchmark start Jul 12, 2021
@jakirkham
Copy link
Member

Well the other interesting thing from that thread was pinning the Scheduler to a single CPU. Admittedly that's probably a little hacky (ok for the exploration Gabe is doing). If that is important longer term, we can investigate other ways to set that.

@gjoseph92
Copy link

@jakirkham worth noting that I only did that so the measured CPU times (particularly idle time) were easier to understand on a multi-core machine. The disabling-GC performance boost seemed the same without it. I suppose it could make a difference with memory access performance if the kernel is moving the process around between physical cores, but I don't think we're at the point of worrying about that yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants