Skip to content

Explain <query> should not fail if meeting errors when optimizing the query #4766

@HaoYang670

Description

@HaoYang670

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently, the explain <query> gives the same error messages as the query if it meets errors in plan optimizations.
For example, the query in #4677 meets an internal error when initializing the physical plan:

select count(*) from (select * from t union all select * from t);
Internal("create_physical_expr expected same number of fields, got got Arrow schema with 1  and DataFusion schema with 0")

But, we evenly can't see the logical plan of it, although there is no error met in the logical optimization

❯ explain select count(*) from (select * from t union all select * from t);
Internal("create_physical_expr expected same number of fields, got got Arrow schema with 1  and DataFusion schema with 0")
❯ explain verbose select count(*) from (select * from t union all select * from t);
Internal("create_physical_expr expected same number of fields, got got Arrow schema with 1  and DataFusion schema with 0")

Describe the solution you'd like
The explain should print the plans until meeting errors. This could help developers to debug.
For the above example, I'd like the explain and explain verbose to print like this:

❯ explain select count(*) from (select * from t union all select * from t);
+---------------+-------------------------------------------------------------+
| plan_type     | plan                                                        |
+---------------+-------------------------------------------------------------+
| logical_plan  | Projection: COUNT(UInt8(1))                                 |
|               |   Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1))]]         |
|               |     Union                                                   |
|               |       TableScan: t projection=[a]                           |
|               |       TableScan: t projection=[a]                           |
| physical_plan | InternalError("...")                                        |
|               |                                                             |
+---------------+-------------------------------------------------------------+

❯ explain verbose select count(*) from (select * from t union all select * from t);
+------------------------------------------------------------+-----------------------------------------------------------------+
| plan_type                                                  | plan                                                            |
+------------------------------------------------------------+-----------------------------------------------------------------+
| initial_logical_plan                                       | Projection: COUNT(UInt8(1))                                     |
|                                                            |   Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1))]]             |
|                                                            |     Union                                                       |
|                                                            |       Projection: t.a, t.b                                      |
|                                                            |         TableScan: t                                            |
|                                                            |       Projection: t.a, t.b                                      |
|                                                            |         TableScan: t                                            |
| logical_plan after inline_table_scan                       | SAME TEXT AS ABOVE                                              |
| logical_plan after type_coercion                           | SAME TEXT AS ABOVE                                              |
| logical_plan after extract_equijoin_predicate              | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after unwrap_cast_in_comparison               | SAME TEXT AS ABOVE                                              |
| logical_plan after decorrelate_where_exists                | SAME TEXT AS ABOVE                                              |
| logical_plan after decorrelate_where_in                    | SAME TEXT AS ABOVE                                              |
| logical_plan after scalar_subquery_to_join                 | SAME TEXT AS ABOVE                                              |
| logical_plan after subquery_filter_to_join                 | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_filter                        | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_cross_join                    | SAME TEXT AS ABOVE                                              |
| logical_plan after common_sub_expression_eliminate         | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_limit                         | SAME TEXT AS ABOVE                                              |
| logical_plan after propagate_empty_relation                | SAME TEXT AS ABOVE                                              |
| logical_plan after rewrite_disjunctive_predicate           | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_outer_join                    | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_limit                         | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_filter                        | SAME TEXT AS ABOVE                                              |
| logical_plan after single_distinct_aggregation_to_group_by | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after unwrap_cast_in_comparison               | SAME TEXT AS ABOVE                                              |
| logical_plan after common_sub_expression_eliminate         | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_projection                    | Projection: COUNT(UInt8(1))                                     |
|                                                            |   Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1))]]             |
|                                                            |     Union                                                       |
|                                                            |       TableScan: t projection=[a]                               |
|                                                            |       TableScan: t projection=[a]                               |
| logical_plan after inline_table_scan                       | SAME TEXT AS ABOVE                                              |
| logical_plan after type_coercion                           | SAME TEXT AS ABOVE                                              |
| logical_plan after extract_equijoin_predicate              | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after unwrap_cast_in_comparison               | SAME TEXT AS ABOVE                                              |
| logical_plan after decorrelate_where_exists                | SAME TEXT AS ABOVE                                              |
| logical_plan after decorrelate_where_in                    | SAME TEXT AS ABOVE                                              |
| logical_plan after scalar_subquery_to_join                 | SAME TEXT AS ABOVE                                              |
| logical_plan after subquery_filter_to_join                 | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_filter                        | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_cross_join                    | SAME TEXT AS ABOVE                                              |
| logical_plan after common_sub_expression_eliminate         | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_limit                         | SAME TEXT AS ABOVE                                              |
| logical_plan after propagate_empty_relation                | SAME TEXT AS ABOVE                                              |
| logical_plan after rewrite_disjunctive_predicate           | SAME TEXT AS ABOVE                                              |
| logical_plan after eliminate_outer_join                    | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_limit                         | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_filter                        | SAME TEXT AS ABOVE                                              |
| logical_plan after single_distinct_aggregation_to_group_by | SAME TEXT AS ABOVE                                              |
| logical_plan after simplify_expressions                    | SAME TEXT AS ABOVE                                              |
| logical_plan after unwrap_cast_in_comparison               | SAME TEXT AS ABOVE                                              |
| logical_plan after common_sub_expression_eliminate         | SAME TEXT AS ABOVE                                              |
| logical_plan after push_down_projection                    | SAME TEXT AS ABOVE                                              |
| logical_plan                                               | Projection: COUNT(UInt8(1))                                     |
|                                                            |   Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1))]]             |
|                                                            |     Union                                                       |
|                                                            |       TableScan: t projection=[a]                               |
|                                                            |       TableScan: t projection=[a]                               |
| initial_physical_plan                                      | InernalError("...")                                             |
|                                                            |                                                                 |
+------------------------------------------------------------+-----------------------------------------------------------------+

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions