Using display_name for Expr::Aggregation #11020
Conversation
|
Also happened to solve part of #10364 |
|
The reason for the error is that the field name generation rules of AggregationFunction are different in logicalplan and physicalplan. The former uses display_name() and the latter uses create_physical_name. |
| @@ -1916,6 +1916,7 @@ pub fn create_aggregate_expr_and_maybe_filter( | |||
| // unpack (nested) aliased logical expressions, e.g. "sum(col) as total" | |||
| let (name, e) = match e { | |||
| Expr::Alias(Alias { expr, name, .. }) => (name.clone(), expr.as_ref()), | |||
| Expr::AggregateFunction(_) => (e.display_name().unwrap_or(physical_name(e)?), e), | |||
There was a problem hiding this comment.
How about we call display_name in physical_name like what ScalarFunction does.
And even more, I think there are no such difference between logical name and physical name, maybe we can have one name for these two 🤔
There was a problem hiding this comment.
I would be nice if we also use display_name in impl fmt::Display for Expr
There was a problem hiding this comment.
It seems CastExpr is different in write_name and Expr's Display 🤔
And, there is an issue that expect CastExpr be shown in write name
CastExpr is removed in #3222, but I didn't find the rationale of doing this
If the issue is due to the schema change, then it will be quite tricky #10276
We can reuse display_name in physical_name to fix the regression first
There was a problem hiding this comment.
It seems CastExpr is different in
write_nameandExpr's Display🤔And, there is an issue that expect CastExpr be shown in
write nameCastExpr is removed in #3222, but I didn't find the rationale of doing this
If the issue is due to the schema change, then it will be quite tricky #10276
We can reuse
display_nameinphysical_nameto fix the regression first
Yes, I think calling display_name like physical name is a good idea, I tried it before, but there are some tests failed, there might be some reason that the display_name is different from physical name, I think for this issue, only change AggregateFunction is enough, I can take other related issue and fix them in the following PR, Thanks for review !
|
|
||
| ctx.register_table("test", Arc::new(table))?; | ||
|
|
||
| let sql = "SELECT MIN(a) FILTER (WHERE a > 1) AS x FROM test"; |
There was a problem hiding this comment.
I'm not sure what is this test for 🤔
Is it possible to move to slt?
statement ok
set datafusion.sql_parser.dialect = 'Postgres';
statement ok
create table t (a float) as values (1), (2), (3);
query TT
explain select min(a) filter (where a > 1) as x from t;
----
logical_plan
01)Projection: MIN(t.a) FILTER (WHERE t.a > Int64(1)) AS x
02)--Aggregate: groupBy=[[]], aggr=[[MIN(t.a) FILTER (WHERE t.a > Float32(1)) AS MIN(t.a) FILTER (WHERE t.a > Int64(1))]]
03)----TableScan: t projection=[a]
physical_plan
01)ProjectionExec: expr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))@0 as x]
02)--AggregateExec: mode=Single, gby=[], aggr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))]
03)----MemoryExec: partitions=1, partition_sizes=[1]
statement ok
drop table t;
There was a problem hiding this comment.
Sure, I can do that.
There was a problem hiding this comment.
I'm not sure what is this test for 🤔
Is it possible to move to slt?
statement ok set datafusion.sql_parser.dialect = 'Postgres'; statement ok create table t (a float) as values (1), (2), (3); query TT explain select min(a) filter (where a > 1) as x from t; ---- logical_plan 01)Projection: MIN(t.a) FILTER (WHERE t.a > Int64(1)) AS x 02)--Aggregate: groupBy=[[]], aggr=[[MIN(t.a) FILTER (WHERE t.a > Float32(1)) AS MIN(t.a) FILTER (WHERE t.a > Int64(1))]] 03)----TableScan: t projection=[a] physical_plan 01)ProjectionExec: expr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))@0 as x] 02)--AggregateExec: mode=Single, gby=[], aggr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))] 03)----MemoryExec: partitions=1, partition_sizes=[1] statement ok drop table t;
This test is for the original issue #10878
There was a problem hiding this comment.
I'm not sure what is this test for 🤔
Is it possible to move to slt?statement ok set datafusion.sql_parser.dialect = 'Postgres'; statement ok create table t (a float) as values (1), (2), (3); query TT explain select min(a) filter (where a > 1) as x from t; ---- logical_plan 01)Projection: MIN(t.a) FILTER (WHERE t.a > Int64(1)) AS x 02)--Aggregate: groupBy=[[]], aggr=[[MIN(t.a) FILTER (WHERE t.a > Float32(1)) AS MIN(t.a) FILTER (WHERE t.a > Int64(1))]] 03)----TableScan: t projection=[a] physical_plan 01)ProjectionExec: expr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))@0 as x] 02)--AggregateExec: mode=Single, gby=[], aggr=[MIN(t.a) FILTER (WHERE t.a > Int64(1))] 03)----MemoryExec: partitions=1, partition_sizes=[1] statement ok drop table t;This test is for the original issue #10878
I guess the test you write only test if the value is correct or not, didn't test if the name is expected
|
An unrelated issue with CI/CD |
|
Thanks @Lordworms |
Which issue does this PR close?
Closes #10878
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?