Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better documentation for timely::execute #293

Closed
ryzhyk opened this issue Sep 5, 2019 · 2 comments · Fixed by #311
Closed

Better documentation for timely::execute #293

ryzhyk opened this issue Sep 5, 2019 · 2 comments · Fixed by #311

Comments

@ryzhyk
Copy link
Contributor

ryzhyk commented Sep 5, 2019

The documentation for timely::execute() does not specify when the function returns. The description says "Starts a timely dataflow execution...", and so I expected the function to return immediately after starting the worker threads. Experimentally, however, it seems to only return when the dataflow is terminating. Is this the intended behavior? If so, it should probably be documented.
@frankmcsherry
Copy link
Member

It should return almost immediately. It spawns several threads with the dataflow assembly instructions and then returns an instance of JoinGuards which wraps handles to each of the worker threads and their possible results. If you drop that result, the drop code will block awaiting the completion. More about that type here: https://github.com/TimelyDataflow/timely-dataflow/blob/master/communication/src/initialize.rs#L277-L306

@ryzhyk
Copy link
Contributor Author

ryzhyk commented Sep 5, 2019 •
edited
Loading

Aha, that was it, I was dropping the WorkerGuards. I still think it is worth documenting this behavior. Will try to write it up and submit a PR.

ryzhyk added a commit to ddlog-dev/differential-datalog that referenced this issue Sep 5, 2019
@ryzhyk
program.rs: Use timely::execute correctly.
a1ccd11 
We use the `timely::execute` API to start the timely computation.  This
function used to be invoked in a separate thread, because I was under
the impression that it does not return until the timely computation
terminates.  Turns out
(TimelyDataflow/timely-dataflow#293), the
function returns instantly, but it returns an instance of
`WorkerGuards`, which, if dropped, blocks the current thread until the
timely computation terminates.

This patch gets rid of the extra thread and the associated ugliness and
instead calls `execute` directly from `Program::run` and holds on to
`WorkerGuards` until it is time to terminate the computation (in
`RunningProgram::stop()`).
frankmcsherry added a commit that referenced this issue Nov 18, 2019
@frankmcsherry
Clarify execute blocking behavior.
e336e83 
fixes #293
@frankmcsherry frankmcsherry mentioned this issue Nov 18, 2019
Merged
saradecova pushed a commit to saradecova/timely-dataflow that referenced this issue Jan 31, 2020
@frankmcsherry @saradecova
Clarify execute blocking behavior.
35590f5 
fixes TimelyDataflow#293
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Clarify execute blocking behavior. TimelyDataflow/timely-dataflow
2 participants
@frankmcsherry @ryzhyk