Two observations have led to this:
- BooleanGrouperTest's use of a case when filter to verify results pushes runners into each branch (due to CSE's ConditionalExpression handling)
- Even when carefully using LCA (as per above) sub queries are only evaluated once per each of these "copies"
Although the case when behaviour will only evaluate once it still causes multiple compilation runs and unnecessary instance creation.
Nondeterministic expressions get special ProjectExec handling due to SPARK-33092 in 3.1.1 and are only evaluated once.
CSE itself will not by default look into the expression tree beneath a nondeterministic expression. The separate compilation introduced in 0.2.0 (normal and grouped) should still perform CSE on the children.
Two observations have led to this:
Although the case when behaviour will only evaluate once it still causes multiple compilation runs and unnecessary instance creation.
Nondeterministic expressions get special ProjectExec handling due to SPARK-33092 in 3.1.1 and are only evaluated once.
CSE itself will not by default look into the expression tree beneath a nondeterministic expression. The separate compilation introduced in 0.2.0 (normal and grouped) should still perform CSE on the children.