Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve mention of aggregate() in the partitionAggregate() docs #2362

Merged
merged 1 commit into from Oct 9, 2017

Conversation

F30
Copy link
Contributor

@F30 F30 commented Oct 8, 2017

This is a revival of #1018: It took a while, but I have now rebased the changes onto the current master and adjusted the base branch.

However, #1018 is in "missing common history" mode after force-pushing the new commits. Changing the base branch didn't cause the diff to be recomputed, so the Pull Request cannot be re-opened.

Therefore, this new PR is required.

@@ -484,7 +484,7 @@ public class Count implements CombinerAggregator<Long> {
}
```

The benefits of CombinerAggregators are seen when you use them with the aggregate method instead of partitionAggregate. In that case, Trident automatically optimizes the computation by doing partial aggregations before transferring tuples over the network.
CombinerAggregators offer high efficiency when used with the aggregate method instead of partitionAggregate (see below).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little unsure which part of the document below this is referring to. Can you insert a link instead, e.g. using https://gist.github.com/asabaylus/3071099?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a link now.

It was not obvious (to me) that aggregate() is covered in detail at
another place in the document. That specific section describes the
details of the optimization much more exhaustively.
Also, it was unclear what the benefits should be compared to, because
other aggregator types are only introduced afterwards.
@srdo
Copy link
Contributor

srdo commented Oct 9, 2017

Thanks, +1

@asfgit asfgit merged commit 38a32f8 into apache:master Oct 9, 2017
@srdo
Copy link
Contributor

srdo commented Oct 9, 2017

Merged to master, 1.x and 1.1.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants