Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support a configurable delay for task shuffles. #1770

Merged
merged 8 commits into from Apr 5, 2018
Merged

Conversation

@baconmania
Copy link
Contributor

@baconmania baconmania commented Mar 29, 2018

No description provided.

@baconmania baconmania changed the title Support a configurable delay for task shuffles. (WIP) Support a configurable delay for task shuffles. Mar 29, 2018
return (
isLongRunning(task)
&& !configuration.getDoNotShuffleRequests().contains(task.getRequestId())
&& (TimeUnit.MILLISECONDS.toMinutes(System.currentTimeMillis() - task.getStartedAt()) > configuration.getMinutesBeforeNewTaskEligibleForShuffle())

This comment has been minimized.

@ssalinas

ssalinas Mar 30, 2018
Member

I wonder if we should go by the task running timestamp instead? Take out any delay that may have been caused by mesos/singularity that way

@ssalinas ssalinas added this to the 0.20.0 milestone Mar 30, 2018
@baconmania baconmania changed the title (WIP) Support a configurable delay for task shuffles. Support a configurable delay for task shuffles. Mar 30, 2018
@@ -15,6 +15,8 @@
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Collectors;

import javax.ws.rs.HEAD;

This comment has been minimized.

@ssalinas

ssalinas Apr 3, 2018
Member

leftover from merge conflict? (intellij auto import stuff?)

This comment has been minimized.

@baconmania

baconmania Apr 3, 2018
Author Contributor

Good catch.

@ssalinas
Copy link
Member

@ssalinas ssalinas commented Apr 3, 2018

🚢

@ssalinas ssalinas merged commit 52bf623 into master Apr 5, 2018
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@ssalinas ssalinas deleted the delayed-task-shuffles branch Apr 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.