refactor(expr): prevent implicit array-compaction in Expression::eval #3523
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: TennyZhuang <zty0826@gmail.com>
TennyZhuang
requested review from
lmatz,
fuyufjh,
xiangjinwu,
BugenZhao,
yuhao-su and
BowenXiao1999
fuyufjh
approved these changes
Jun 28, 2022
src/expr/src/expr/mod.rs
Outdated
| @@ -56,6 +56,14 @@ pub type ExpressionRef = Arc<dyn Expression>; | |||
| pub trait Expression: std::fmt::Debug + Sync + Send { | |||
| fn return_type(&self) -> DataType; | |||
|
|
|||
| fn wrapping_eval(&self, input: &DataChunk) -> Result<ArrayRef> { | |||
src/expr/src/expr/mod.rs
Outdated
| @@ -128,7 +136,7 @@ impl RowExpression { | |||
|
|
|||
| pub fn eval(&mut self, row: &Row, data_types: &[DataType]) -> Result<ArrayRef> { | |||
| let input = DataChunk::from_rows(slice::from_ref(row), data_types)?; | |||
| self.expr.eval(&input) | |||
| self.expr.wrapping_eval(&input) | |||
There was a problem hiding this comment.
eval is the implementation, and wrapping_eval add some common checks.
There was a problem hiding this comment.
wrapping sounds like a numeric overflow wrapping to me. 😢 Name it checked_eval or something?
There was a problem hiding this comment.
wrapping is just eval with a wrapper :)
Currently we do check here, but I'm thinking about something else, such as tracing span, and the check may also be updated to debug_assert! if we have some benchmarks.
BugenZhao
reviewed
Jun 29, 2022
src/expr/src/expr/mod.rs
Outdated
| @@ -128,7 +136,7 @@ impl RowExpression { | |||
|
|
|||
| pub fn eval(&mut self, row: &Row, data_types: &[DataType]) -> Result<ArrayRef> { | |||
| let input = DataChunk::from_rows(slice::from_ref(row), data_types)?; | |||
| self.expr.eval(&input) | |||
| self.expr.wrapping_eval(&input) | |||
There was a problem hiding this comment.
wrapping sounds like a numeric overflow wrapping to me. 😢 Name it checked_eval or something?
src/expr/src/expr/mod.rs
Outdated
| @@ -56,6 +56,14 @@ pub type ExpressionRef = Arc<dyn Expression>; | |||
| pub trait Expression: std::fmt::Debug + Sync + Send { | |||
| fn return_type(&self) -> DataType; | |||
|
|
|||
| fn wrapping_eval(&self, input: &DataChunk) -> Result<ArrayRef> { | |||
Signed-off-by: TennyZhuang <zty0826@gmail.com>
xiangjinwu
approved these changes
Jun 29, 2022
| fn wrapping_eval(&self, input: &DataChunk) -> Result<ArrayRef> { | ||
| let res = self.eval(input)?; | ||
|
|
||
| assert_eq!(res.len(), input.capacity()); |
There was a problem hiding this comment.
off topic: another useful check would be that array variant (ArrayImpl::Int32) matches expected return type (DataType::Int32).
Signed-off-by: TennyZhuang <zty0826@gmail.com>
Signed-off-by: TennyZhuang <zty0826@gmail.com>
BugenZhao
approved these changes
Jun 29, 2022
TennyZhuang
added
the
mergify/can-merge
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Codecov Report
@@ Coverage Diff @@
## main #3523 +/- ##
==========================================
- Coverage 74.45% 74.45% -0.01%
==========================================
Files 770 770
Lines 108455 108517 +62
==========================================
+ Hits 80750 80792 +42
- Misses 27705 27725 +20
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more |
huangjw806
pushed a commit
that referenced
this pull request
Jul 5, 2022
…#3523) * refactor(expr): prevent implicit array-compaction in Expression::eval Signed-off-by: TennyZhuang <zty0826@gmail.com> * address comment Signed-off-by: TennyZhuang <zty0826@gmail.com> * add comments Signed-off-by: TennyZhuang <zty0826@gmail.com> * rename to eval_checked Signed-off-by: TennyZhuang <zty0826@gmail.com> * fix test Signed-off-by: TennyZhuang <zty0826@gmail.com> * fix clippy Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: TennyZhuang zty0826@gmail.com
I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
PLEASE DO NOT LEAVE THIS EMPTY !!!
As title
It's still error-prone, but I can't find a better solution now.
Checklist
./risedev check(or alias,./risedev c)Refer to a related PR or issue link (optional)