[CALCITE-6667] Arrow adapter should push down arithmetic operations#4031
[CALCITE-6667] Arrow adapter should push down arithmetic operations#4031caicancai wants to merge 2 commits intoapache:mainfrom
Conversation
…action operations
|
Currently, only simple constant addition and subtraction operations are supported. I did not perform complex type tests because there are some other problems with the current adaptation of the arrow type. |
| case PLUS: | ||
| right = translateArithmeticOp(left, right); | ||
| return translateBinary2(op, ((RexCall) left).operands.get(0), right); | ||
| case MINUS: |
There was a problem hiding this comment.
these two cases could be merged
caicancai
changed the title
|
| * @return A new RexNode representing the adjusted right operand | ||
| */ | ||
| private RexNode translateArithmeticOp(RexNode left, RexNode right) { | ||
| RexNode leftOperand = ((RexCall) left).operands.get(1); |
There was a problem hiding this comment.
This optimization is already part of PROJECT_REDUCE_EXPRESSIONS. I don't think it should be replicated in any specific backend.
There was a problem hiding this comment.
I'm not sure if it works, and it seems no one has tried it yet.
There was a problem hiding this comment.
I don't know what you mean, in RelOptRulesTest there are at least 59 tests that use this optimization.
There was a problem hiding this comment.
I understand what you mean, but it seems that no other adapters are specified for testing. I understand that RelOptRulesTest is for those Jdbc protocol dialects, such as mysql, postgres and so on. I am not sure whether adapters such as arrow, es, etc. are effective.
There was a problem hiding this comment.
That is incorrect, the RelOptRules are designed to work for all dialects.
A Calcite-based compiler will have a parser, a converter, an optimizer, and a back-end.
The job of doing optimizations belongs to the optimizer, not to the back-end.
If you put optimizations in the back-end, you have to reimplement them for every target.
There was a problem hiding this comment.
Thank you for your answer, so I just need to translate the plan without any optimization, I will improve this pr later
There was a problem hiding this comment.
@mihaibudiu It seems that Arrow's API does not support operations like "intField\" + 1 = 12. I had to convert it to "intField\" = 11 in Calcite.
Should I call the PROJECT_REDUCE_EXPRESSIONS optimization rule in the arrow adapter?
There was a problem hiding this comment.
Adapters should never call optimizations.
This particular optimization does constant folding, and the expression you show does not have any constant subexpressions, so this optimization will not help.
2 participants
https://issues.apache.org/jira/browse/CALCITE-6667