scx_rustland: prevent dispatching multiple tasks on the same idle cpu #58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a task is dispatched we always try to pick the previously used CPU (if idle) to minimize the migration overhead. Alternatively, if such CPU is not available, we pick any other idle CPU in the system. However, we don't update the list of idle CPUs as we dispatch tasks, therefore we may end up sending multiple tasks to the same idle CPU (if their previously used CPU is the same) and we may even skip some idle CPUs completely. Change this logic to make sure that we never dispatch multiple tasks to the same idle CPU, by updating the list of idle CPUs as we send tasks to the BPF dispatcher. This also avoids dispatching tasks with a closely matched vruntime to the same CPU, thereby negating the advantages of the vruntime ordering. With this change in place, we ensure that tasks with a similar vruntime are dispatched to different CPUs, leading to significant improvements in latency performance. Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
htejun
approved these changes
Dec 31, 2023
htejun
approved these changes
Dec 31, 2023
| @@ -406,8 +406,14 @@ impl<'a> Scheduler<'a> { | |||
| // | |||
| // Use the previously used CPU if idle, that is always the best choice (to | |||
| // mitigate migration overhead), otherwise pick the next idle CPU available. | |||
| if !idle_cpus.contains(&task.cpu) { | |||
| task.cpu = *cpu; | |||
| if let Some(id) = idle_cpus.iter().position(|&x| x == task.cpu) { | |||
There was a problem hiding this comment.
Bike-shedding but I find it a little confusing to receive position as id. Maybe something like idx, i or pos is better?
There was a problem hiding this comment.
@htejun yeah it is confusing, id sounds more like the value in this context, not the position, pos is a lot more clear.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a task is dispatched we always try to pick the previously used CPU (if idle) to minimize the migration overhead. Alternatively, if such CPU is not available, we pick any other idle CPU in the system.
However, we don't update the list of idle CPUs as we dispatch tasks, therefore we may end up sending multiple tasks to the same idle CPU (if their previously used CPU is the same) and we may even skip some idle CPUs completely.
Change this logic to make sure that we never dispatch multiple tasks to the same idle CPU, by updating the list of idle CPUs as we send tasks to the BPF dispatcher.
This also avoids dispatching tasks with a closely matched vruntime to the same CPU, thereby negating the advantages of the vruntime ordering. With this change in place, we ensure that tasks with a similar vruntime are dispatched to different CPUs, leading to significant improvements in latency performance.