Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a current_tasks argument for retransmission in get_work #1341

Merged
merged 1 commit into from Nov 6, 2015

Conversation

daveFNbuck
Copy link
Contributor

Sometimes a worker will not receive the result of a get_work call from the
scheduler. This leaves the scheduler thinking that the worker is running tasks
that the worker is not running. In order to safely clear up the misunderstanding
we let the scheduler know which tasks the worker is running with each get_work
call. If the scheduler finds that the worker isn't aware of a task that it
should be running, the scheduler will send it to the worker.

In order to maintain backward compatibility and not have to rewrite every unit
test, leaving out current_tasks from the get_work call will be treated as
equivalent to sending every currently running task.

Note that this does not check that every task sent in the currently running
list is actually running.

Sometimes a worker will not receive the result of a get_work call from the
scheduler. This leaves the scheduler thinking that the worker is running tasks
that the worker is not running. In order to safely clear up the misunderstanding
we let the scheduler know which tasks the worker is running with each get_work
call. If the scheduler finds that the worker isn't aware of a task that it
should be running, the scheduler will send it to the worker.

In order to maintain backward compatibility and not have to rewrite every unit
test, leaving out current_tasks from the get_work call will be treated as
equivalent to sending every currently running task.

Note that this does not check that every task sent in the currently running
list is actually running.
erikbern added a commit that referenced this pull request Nov 6, 2015
Adds a current_tasks argument for retransmission in get_work
@erikbern erikbern merged commit c490432 into spotify:master Nov 6, 2015
@daveFNbuck daveFNbuck deleted the get_missed_work branch November 6, 2015 16:15
@Tarrasch
Copy link
Contributor

If the scheduler finds that the worker isn't aware of a task that it
should be running, the scheduler will send it to the worker.

This is a very clever solution! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants