rusty: Rework deadline as a signed sum #309
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Byte-Lab
force-pushed
the
rusty_improved_dl
branch
2 times, most recently
from
8a22d24
to
9198488
Compare
|
Multiple pushes was from adjusting the |
arighi
approved these changes
May 23, 2024
| task_dcycle = 0; | ||
| else | ||
| task_dcycle = DL_FULL_DCYCLE - task_dcycle; | ||
| dcycle_linear = bpf_log2l(max(task_dcycle, 1)); |
There was a problem hiding this comment.
task_dcycle is actually the inverse duty cycle here, or the "idle duty cycle" IIUC, maybe we should rename the variable (i.e., inv_dcycle_linear?) or add a comment to clarify this?
multics69
approved these changes
May 23, 2024
There was a problem hiding this comment.
Thank you for the fantastic work! Reflecting duty cycles maches well with my observation in EEVDF and LAVD. While LAVD does not directly consider the duty cycle, after adding eligibility, I found that LAVD works better under highly loaded system. Since CPU hoggers are mostly ineligible, eligibility might have similar effect to duty cycle. However, the duty cycle is more direct measurement so I like it!
|
I ran the changes in my usual test case. It seems to have a regression. |
|
What I did to trigger the above is as follow:
|
htejun
approved these changes
May 23, 2024
Byte-Lab
force-pushed
the
rusty_improved_dl
branch
from
9198488
to
5ba14c0
Compare
|
Pushed code that addresses @arighi and @multics69's feedback. Going to take a look at the regression that Changwoo pointed out before merging. |
|
Incredible works you guys are doing , I'm just here to learn what you changed and the reasons. |
|
Actually, this PR seems to introduce some regression for some users in a specific workload. The user did following:
Following version has been used for testing:
Here the video can be found. There is a quite huge interactivity/FPS drop visible in Rocket Leauge: |
Currently, a task's deadline is computed as its vtime + a scaled function of
its average runtime (with its deadline being scaled down if it's more
interactive). This makes sense intuitively, as we do want an interactive task
to have an earlier deadline, but it also has some flaws.
For one thing, we're currently ignoring duty cycle when determining a task's
deadline. This has a few implications. Firstly, because we reward tasks with
higher waker and blocked frequencies due to considering them to be part of a
work chain, we implicitly penalize tasks that rarely ever use the CPU because
those frequencies are low. While those tasks are likely not part of a work
chain, they also should get an interactivity boost just by pure virtue of not
using the CPU very often. This should in theory be addressed by vruntime, but
because we cap the amount of vtime that a task can accumulate to one slice, it
may not be adequately reflected after a task runs for the first time.
Another problem is that we're minimizing a task's deadline if it's interactive,
but we're also not really penalizing a task that's a super CPU hog by
increasing its deadline. We sort of do a bit by applying a higher niceness
which gives it a higher deadline for a lower weight, but its somewhat minimal
considering that we're using niceness, and that the best an interactive task
can do is minimize its deadline to near zero relative to its vtime.
What we really want to do is "negatively" scale an interactive task's deadline
with the same magnitude as we "positively" scale a CPU-hogging task's deadline.
To do this, we make two major changes to how we compute deadline:
1. Instead of using niceness, we now instead use our own straightforward
scaling factor. This was chosen arbitrarily to be a scaling by 1000, but we
can and should improve this in the future.
2. We now create a _signed_ linear latency priority factor as a sum of the
three following inputs:
- Work-chain factor (log_2 of product of blocked freq and waker freq)
- Inverse duty cycle factor (log_2 of the inverse of a task's duty cycle --
higher duty cycle means lower factor)
- Average runtime factor (Higher avg runtime means higher average runtime
factor)
We then compute the latency priority as:
lat_prio := Average runtime factor - (work-chain factor + duty cycle factor)
This gives us a signed value that can be negative. With this, we can compute a
non-negative weight value by calculating a weight from the absolute value of
lat_prio, and use this to scale slice_ns. If lat_prio is negative we calculate
a task's deadline as its vtime MINUS its scaled slice_ns, and if it's positive,
it's the task's vtime PLUS scaled slice_ns.
This ends up working well because you get a higher weight both for highly
interactive tasks, and highly CPU-hogging / non-interactive tasks, which lets
you scale a task's deadline "more negatively" for interactive tasks, and "more
positively" for the CPU hogs.
With this change, we get a significant improvement in FPS. On a 7950X, if I run
the following workload:
$ stress-ng -c $((8 * $(nproc)))
1. I get 60 FPS when playing Stellaris (while time is progressing at max
speed), whereas EEVDF gets 6-7 FPS.
2. I get ~15-40 FPS while playing Civ6, whereas EEVDF seems to get < 1 FPS. The
Civ6 benchmark doesn't even start after over 4 minutes in the initial frame
with EEVDF, but gets us 13s / turn with rusty.
3. It seems that EEVDF has improved with Terraria in v6.9. It was able to
maintain ~30-55 FPS, as opposed to the ~5-10FPS we've seen in the past.
rusty is still able to maintain a solid 60-62FPS consistently with no
problem, however.
…e tasks In some scenarios, a CPU-intensive task may be on the critical path for interactive workloads. For example, you may have a game with CPU-intensive tasks that are crunching the logic for the game, and that's required for the game to proceed without being choppy. To support such workflows, this change adds logic to allow a non-interactive task to inherit the lower (i.e. stronger) latency priority of another task if it wakes or is woken by that task. Signed-off-by: David Vernet <void@manifault.com>
Byte-Lab
force-pushed
the
rusty_improved_dl
branch
from
5ba14c0
to
c1ad602
Compare
|
Alright, this seems to beat the old rusty on RocketLeague now as well. Interestingly, EEVDF on v6.10 beats both versions of rusty on RocketLeague, but still gets trounced on Terraria and Civ6. On Civ6 especially the game becomes unplayable with EEVDF under heavy background load, but is still fairly responsive with rusty. This isn't perfect, but I'm going to merge it for now and we can continue to iterate in tree. |
|
Getting reports that this causes some regressions in cachy, so lemme revert while we investigate. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, a task's deadline is computed as its vtime + a scaled function of its average runtime (with its deadline being scaled down if it's more interactive). This makes sense intuitively, as we do want an interactive task to have an earlier deadline, but it also has some flaws.
For one thing, we're currently ignoring duty cycle when determining a task's deadline. This has a few implications. Firstly, because we reward tasks with higher waker and blocked frequencies due to considering them to be part of a work chain, we implicitly penalize tasks that rarely ever use the CPU because those frequencies are low. While those tasks are likely not part of a work chain, they also should get an interactivity boost just by pure virtue of not using the CPU very often. This should in theory be addressed by vruntime, but because we cap the amount of vtime that a task can accumulate to one slice, it may not be adequately reflected after a task runs for the first time.
Another problem is that we're minimizing a task's deadline if it's interactive, but we're also not really penalizing a task that's a super CPU hog by increasing its deadline. We sort of do a bit by applying a higher niceness which gives it a higher deadline for a lower weight, but its somewhat minimal considering that we're using niceness, and that the best an interactive task can do is minimize its deadline to near zero relative to its vtime.
What we really want to do is "negatively" scale an interactive task's deadline with the same magnitude as we "positively" scale a CPU-hogging task's deadline. To do this, we make two major changes to how we compute deadline:
Instead of using niceness, we now instead use our own straightforward scaling factor. This was chosen arbitrarily (by experimentation with some games) to be a scaling by 1000, but we can and should improve this in the future.
We now create a signed linear latency priority factor as a sum of the three following inputs:
We then compute the latency priority as:
This gives us a signed value that can be negative. With this, we can compute a non-negative weight value by calculating a weight from the absolute value of lat_prio, and use this to scale slice_ns. If lat_prio is negative we calculate a task's deadline as its vtime MINUS its scaled slice_ns, and if it's positive, it's the task's vtime PLUS scaled slice_ns.
This ends up working well because you get a higher weight both for highly interactive tasks, and highly CPU-hogging / non-interactive tasks, which lets you scale a task's deadline "more negatively" for interactive tasks, and "more positively" for the CPU hogs.
With this change, we get a significant improvement in FPS. On a 7950X, if I run the following workload:
I get 60 FPS when playing Stellaris (while time is progressing at max speed), whereas EEVDF gets 6-7 FPS.
I get ~15-40 FPS while playing Civ6, whereas EEVDF seems to get < 1 FPS. The Civ6 benchmark doesn't even start after over 4 minutes in the initial frame with EEVDF, but gets us 13s / turn with rusty.
It seems that EEVDF has improved with Terraria in v6.9. It was able to maintain ~30-55 FPS, as opposed to the ~5-10FPS we've seen in the past. rusty is still able to maintain a solid 60-62FPS consistently with no problem, however.