Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(streaming): remove dist key for global simple agg & rename to GlobalSimpleAgg #3548

Merged
merged 6 commits into from
Jun 29, 2022

Conversation

BugenZhao
Copy link
Member

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

As title. We reuse SimpleAggNode for both global and local ones, where dist_key should be unused for global simple agg.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@@ -56,6 +56,7 @@ message MaterializeNode {
// Local and global aggregator distinguish with each other in PlanNode definition.
message SimpleAggNode {
repeated expr.AggCall agg_calls = 1;
// Not used for global simple agg.
repeated uint32 distribution_keys = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not deleted?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SimpleAggNode is also used for local agg. 🤣

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao BugenZhao requested a review from kwannoel June 29, 2022 06:51
Copy link
Contributor

@kwannoel kwannoel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for doing this refactor :) Missed this in earlier PR.

Copy link
Contributor

@BowenXiao1999 BowenXiao1999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LSTM

BTW can we also change the name from "StreamSimpleAgg" to "StreamGlobalSimpleAgg" in this PR? 😢 https://github.com/singularity-data/risingwave/pull/3320/files

@codecov
Copy link

codecov bot commented Jun 29, 2022

Codecov Report

Merging #3548 (42e486b) into main (3201d61) will increase coverage by 0.00%.
The diff coverage is 87.50%.

@@           Coverage Diff           @@
##             main    #3548   +/-   ##
=======================================
  Coverage   74.44%   74.44%           
=======================================
  Files         770      770           
  Lines      108424   108409   -15     
=======================================
- Hits        80715    80707    -8     
+ Misses      27709    27702    -7     
Flag Coverage Δ
rust 74.44% <87.50%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/frontend/src/optimizer/plan_node/mod.rs 97.91% <ø> (ø)
src/stream/src/executor/aggregation/agg_state.rs 82.97% <ø> (ø)
src/stream/src/executor/mod.rs 51.22% <ø> (ø)
src/stream/src/from_proto/global_simple_agg.rs 0.00% <0.00%> (ø)
src/stream/src/from_proto/mod.rs 0.00% <ø> (ø)
...rc/frontend/src/optimizer/plan_node/logical_agg.rs 92.92% <100.00%> (ø)
...rc/optimizer/plan_node/stream_global_simple_agg.rs 96.66% <100.00%> (ø)
src/stream/src/executor/global_simple_agg.rs 95.81% <100.00%> (-0.04%) ⬇️
src/stream/src/executor/test_utils.rs 95.03% <100.00%> (-0.21%) ⬇️
src/meta/src/manager/id.rs 96.06% <0.00%> (+0.56%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@BugenZhao
Copy link
Member Author

StreamSimpleAgg

Cool.

@BugenZhao BugenZhao changed the title refactor(streaming): remove dist key for global simple agg & rename to GlobalSimpleAggExecutor refactor(streaming): remove dist key for global simple agg & rename to GlobalSimpleAgg Jun 29, 2022
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao BugenZhao force-pushed the bz/remove-dist-key-in-global-simple-agg branch from d4dce3f to 1432209 Compare June 29, 2022 07:15
@BugenZhao BugenZhao added the mergify/can-merge Indicates that the PR can be added to the merge queue label Jun 29, 2022
@mergify mergify bot merged commit 9815936 into main Jun 29, 2022
@mergify mergify bot deleted the bz/remove-dist-key-in-global-simple-agg branch June 29, 2022 08:00
huangjw806 pushed a commit that referenced this pull request Jul 5, 2022
…o `GlobalSimpleAgg` (#3548)

* remove dist key for global simple agg

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

* rename to global

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

* refine proto doc

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

* refine comments

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

* rename frontend plannode

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergify/can-merge Indicates that the PR can be added to the merge queue type/refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants