New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sequencer: order by sequence, use strong references #224

Merged
merged 4 commits into from Feb 4, 2019

Conversation

Projects
None yet
2 participants
@frankmcsherry
Copy link
Member

frankmcsherry commented Feb 4, 2019

This PR introduces to Sequencer worker-local sequence numbers to ensure that the worker-local sequence is reflected in the output. Previously, any batched elements would be introduced at the same time and the sequencer would sort them to ensure a consistent order. Now, each worker tags each item with its index and an increasing counter, and the total order is by (time, worker, counter).

The PR also changes the activator handles to be strong references rather than weak references, to avoid crashes in the case that the Sequencer is dropped before it is drained. This can happen for workers that only plan to react in-dataflow to sequenced messages, and just need to participate in the sequencing (activating themselves to confirm receipt of messages).

frankmcsherry added some commits Feb 4, 2019

@frankmcsherry

This comment has been minimized.

Copy link
Member Author

frankmcsherry commented Feb 4, 2019

@comnik Take a peek and see if this seems sane to you. Although "breaking" it should mostly be repairing things, but let me know if it works for you, and if the intended behavior seems sane for your cases (and generally).

frankmcsherry added some commits Feb 4, 2019

@frankmcsherry frankmcsherry merged commit 06e5cd1 into master Feb 4, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@comnik

This comment has been minimized.

Copy link
Contributor

comnik commented Feb 4, 2019

Great, thanks! Testing with 3DF now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment