Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(stream): implement LocalSimpleAgg without AggExecutor #1823

Merged
merged 6 commits into from
Apr 16, 2022

Conversation

TennyZhuang
Copy link
Collaborator

Signed-off-by: TennyZhuang zty0826@gmail.com

What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

We can try to remove AggExecutorWrapper in executor_v2, due to the following reasons:

  1. The AggExecutorWrapper almost do nothing in executor_v2.
  2. We need to split chunks in HashAgg, the original AggExecutor will not work.
  3. The implementation based on the original AggExecutorWrapper still looks like a state machine.

We may investigate how to reduce the common codes in executor_v2 later.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

Signed-off-by: TennyZhuang <zty0826@gmail.com>
Copy link
Contributor

@pleiadesian pleiadesian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BugenZhao
Copy link
Member

3. The implementation based on the original AggExecutorWrapper still looks like a state machine.

Most migrated executors are simply moved to the new directory and the only changes might be the input type currently. I believe a deep review or rewrite might be required.

Signed-off-by: TennyZhuang <zty0826@gmail.com>
@TennyZhuang TennyZhuang enabled auto-merge (squash) April 14, 2022 10:48
@codecov
Copy link

codecov bot commented Apr 16, 2022

Codecov Report

Merging #1823 (585e093) into main (2458c0e) will decrease coverage by 0.01%.
The diff coverage is 98.14%.

@@            Coverage Diff             @@
##             main    #1823      +/-   ##
==========================================
- Coverage   70.72%   70.70%   -0.02%     
==========================================
  Files         611      611              
  Lines       79896    79861      -35     
==========================================
- Hits        56503    56469      -34     
+ Misses      23393    23392       -1     
Flag Coverage Δ
rust 70.70% <98.14%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/stream/src/executor_v2/local_simple_agg.rs 96.66% <98.14%> (-1.01%) ⬇️
src/connector/src/filesystem/file_common.rs 80.88% <0.00%> (+0.88%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@TennyZhuang TennyZhuang merged commit ebdb341 into main Apr 16, 2022
@TennyZhuang TennyZhuang deleted the refactor/simplify-local-simple-agg branch April 16, 2022 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants