Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inefficient scheduling #68

Open
maharjun opened this issue Jun 16, 2017 · 0 comments
Open

Inefficient scheduling #68

maharjun opened this issue Jun 16, 2017 · 0 comments

Comments

@maharjun
Copy link
Contributor

When futures are spawned on a worker (called source worker), all but 1 (or until time < highwatermark) are sent to the broker. However, if there was a previously running job on this source worker, there would also be a pending task request from this worker at the broker queue. This means that of the spawned futures, it is highly likely (especially if #jobs = #workers) that 2 are executed on the source worker even when they could have been executed separately. for simulations that are small and numerous, this makes little difference, but for simulations that are long, this doubles the execution time.

I propose the following fix (will be implemented in pull request #67):

  1. Send ALL futures spawned to the broker
  2. Maintain an explicit one-one correspondence between requests sent and futures received. i.e. do not spam future requests.
maharjun added a commit to IGITUGraz/scoop that referenced this issue Jun 16, 2017
@maharjun
MULTIPLE CHANGES Fixed Inefficient Scheduling (Issue soravux#68)
8de3f0c 
Made the following changes

1.  Added a new function append_init which is used by futures.submit.
    This function adds newly spawned futures to the broker's task
    queue.

2.  Added new state variable request_in_process to the FutureQueue
    object in order to track the state of the future request. This is
    used to track the state of a future request so that a future
    request is sent iff the previous future request has been answered

3.  Also, replaced function recvFutures with recvIncoming which passes
    the received incoming messages back to updateQueue where they are
    processed. Changed _recv to only perform preprocessing of the
    messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@maharjun