Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch arrange #27

Merged
merged 7 commits into from
Apr 11, 2017
Merged

Batch arrange #27

merged 7 commits into from
Apr 11, 2017

Conversation

frankmcsherry
Copy link
Member

This PR does several things, including some fairly serious business changes to the Trace traits.

The original intent was to allow arrange to accept batches of inputs, allowing us to re-use these batches in the sorting process. This thread lead to several other changes, including a refactoring of the existing trace implementations into common Spine and Batcher generic implementations, and much simpler HashVal and HashKey implementations in implementations/hash.rs. This unification should simplify maintaining all of these implementations.

Testing these changes revealed a bug in join, issue #26, which was fixed up by extending Trace's method cursor to take a frontier, and only iterate over updates that are not in the future of this frontier. This makes join more correct, and has the potential to improve group's implementation in the future. In the meantime, join's implementation is much simpler, and transiently more correct.

@frankmcsherry frankmcsherry merged commit 3477954 into master Apr 11, 2017
@frankmcsherry frankmcsherry deleted the batch_arrange branch April 11, 2017 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant