-
Notifications
You must be signed in to change notification settings - Fork 526
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(streaming): integrate append-only executors to stream graph #3228
Conversation
Codecov Report
@@ Coverage Diff @@
## main #3228 +/- ##
==========================================
- Coverage 73.49% 73.48% -0.01%
==========================================
Files 745 745
Lines 101959 101947 -12
==========================================
- Hits 74930 74916 -14
- Misses 27029 27031 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more |
let mut builder = if self.input().append_only() { | ||
f.debug_struct("StreamAppendOnlyTopN") | ||
} else { | ||
f.debug_struct("StreamTopN") | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks duplicated with "append_only = true/false" in L62~64
(Similar for other operators)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The append_only
attribute in PlanBase
indicates whether the output of operator is append-only, it doesn't mean the operator is append-only optimized. So if we want to check whether the append-only version operator is used, we should differentiate it by name (likes the StreamDeltaJoin
and StreamHashJoin
). cc @st1page
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it. In this way, I would recommend to remove "append_only = true/false" in explain results.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed. I got it wrong before.
Still no e2e tests? 🤣 Maybe add next time. |
Actually, I wrote some e2e tests locally (create append only tables, insert some data, create mv on that table). But I found it is not quite useful. Since non append-only executors can also output the same result. However, now I think it can check that the implementation of append-only executors is correct, I will add a basic test later. |
What's changed and what's your intention?
Integrate append-only version executors (TopN, HashAgg, HashJoin, SimpleAgg) to the stream graph.
Checklist
./risedev check
(or alias,./risedev c
)Refer to a related PR or issue link (optional)
close #2407
close #1550