New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider worker virtualization layer #193

Open
frankmcsherry opened this Issue Oct 3, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@frankmcsherry
Owner

frankmcsherry commented Oct 3, 2018

Timely has workers exactly mapped to threads, but we could have a virtualization layer so that threads can host multiple (or fewer) logical workers.

There are a few reasons to do this, none of which are especially new:

  1. By over-sharding, migration and parallel recovery become easier.
  2. By always using powers of two workers, we can always use masks in data exchange.

There may be some other reasons, who knows, but at least we should talk through what is required for this to happen.

@utaal @antiguru

@frankmcsherry

This comment has been minimized.

Show comment
Hide comment
@frankmcsherry

frankmcsherry Oct 3, 2018

Owner

Some more thoughts: it seems like nothing in the current timely interface exposes the concept of a "thread", but instead asks for per-worker logic. This logic has type

F: Fn(&mut Worker<Allocator>) -> ...

and so it should be instantiable multiple times, even in the same thread.

Now, this is actually a terrible idea, because the code is written as if a single thread that may spin on various resources, rather than one thread that may be driving multiple workers. Without some async/await magic, there isn't a great way to interrupt one of these closures and move to another on the same system thread. Having lots of actual system threads would be a bit of a departure from the current model (though, .. possible).

So, there may need to be a bit of a break in the way we structure these programs, either to patch in multiple workers underneath a call to worker.step(), or a way to yield between multiple closures in a way that is somehow clear and intuitive (hah!).

Owner

frankmcsherry commented Oct 3, 2018

Some more thoughts: it seems like nothing in the current timely interface exposes the concept of a "thread", but instead asks for per-worker logic. This logic has type

F: Fn(&mut Worker<Allocator>) -> ...

and so it should be instantiable multiple times, even in the same thread.

Now, this is actually a terrible idea, because the code is written as if a single thread that may spin on various resources, rather than one thread that may be driving multiple workers. Without some async/await magic, there isn't a great way to interrupt one of these closures and move to another on the same system thread. Having lots of actual system threads would be a bit of a departure from the current model (though, .. possible).

So, there may need to be a bit of a break in the way we structure these programs, either to patch in multiple workers underneath a call to worker.step(), or a way to yield between multiple closures in a way that is somehow clear and intuitive (hah!).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment