Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upLimit the number of tasks that can shuffle for cpu rebalance at once #1751
Conversation
ssalinas
added
hs_staging
hs_qa
labels
Mar 1, 2018
This comment has been minimized.
This comment has been minimized.
|
|
ssalinas
added some commits
Mar 1, 2018
SingularityService/src/main/java/com/hubspot/singularity/scheduler/SingularityUsagePoller.java
Outdated
| @@ -234,7 +246,12 @@ public void runActionOnPoll() { | ||
| if (slaveOverloaded && configuration.isShuffleTasksForOverloadedSlaves()) { | ||
| possibleTasksToShuffle.sort((u1, u2) -> Double.compare(u2.getUsage().getCpusUsed(), u1.getUsage().getCpusUsed())); | ||
| for (TaskIdWithUsage taskIdWithUsage : possibleTasksToShuffle) { | ||
| if (cpuOverage <= 0 || shuffledTasks > configuration.getMaxTasksToShuffleForCpuOverage()) { | ||
| if (requestsWithShuffledTasks.contains(taskIdWithUsage.getTaskId().getRequestId())) { | ||
| LOG.debug("Request {} laready has a shuffling task, skipping", taskIdWithUsage.getTaskId().getRequestId()); |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
|
ssalinas
added
the
hs_stable
label
Mar 1, 2018
ssalinas
added some commits
Mar 1, 2018
This comment has been minimized.
This comment has been minimized.
|
|
This comment has been minimized.
This comment has been minimized.
|
Going to merge this one and start other changes in a new PR |
ssalinas
merged commit 451716b
into
master
Mar 6, 2018
ssalinas
deleted the
shuffle_limit
branch
Mar 6, 2018
ssalinas
added this to the 0.20.0 milestone
Mar 6, 2018
ssalinas
modified the milestones:
0.20.0,
0.19.2
Mar 26, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
ssalinas commentedMar 1, 2018
Add a cluster-wide limit as well as a per-slave limit for how many tasks we can shuffle around at once