Set the scheduling policy individually per thread #135
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kakra
force-pushed
the
feature/thread-scheduling
branch
from
3939ae0
to
84f83c9
Compare
kakra
force-pushed
the
feature/thread-scheduling
branch
from
84f83c9
to
441d4af
Compare
kakra
referenced
this pull request
Nov 1, 2021
Fixes: commit 20b8f8a ("bees: use helper function for readahead") Signed-off-by: Kai Krakow <kai@kaishome.de>
Instead of launching the whole process under one single scheduling policy, let's better set these individually per thread. The crawler, writeback, and prefetch threads will use SCHED_IDLE as we only want them to use CPU when we have it available. The progress and status threads use SCHED_OTHER as we want them to spool their updates per best-effort. Signed-off-by: Kai Krakow <kai@kaishome.de>
This refines the crawler threads which are actually two parts: The subvolume crawler threads will use SCHED_BATCH if available to better utilize CPU caches during number crunching: The kernel will give those threads time slices of 1.5s and will only migrate them to another core if really needed (e.g., if another process needs the core). The crawler master will use SCHED_IDLE as we only want it to use CPU when we have it available. It is only there to generate new work for its sibling threads. Signed-off-by: Kai Krakow <kai@kaishome.de>
We want the loadavg calculation to be almost real-time. If SCHED_ISO is not supported, it will use whatever the service was started with (SCHED_OTHER by default). Signed-off-by: Kai Krakow <kai@kaishome.de>
kakra
force-pushed
the
feature/thread-scheduling
branch
from
441d4af
to
054708d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This removes the CPU scheduler setting from the systemd service and instead let's bees manage this on its own and individually per thread. That way, people not using the systemd service can also benefit from less system impact while using bees.
I'm not sure if we should provide a cmdline option for this. If we do so, maybe just an on/off switch. The user should not have to know the individual best settings.
Also, I'm not sure if I did everything right. The worker/consumer architecture forced me to constantly switch parameters because worker threads are constantly recycled to do a different bees task. Only "BeesThread" seems to be an exception. I had to take care not having a child thread linger around with the scheduler settings of the parent thread. Also, the decision for one or another scheduling policy may be wrong, please review.