You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Certain queries would benefit from materializing part of the query (even if only in memory) and re-planning the rest.
Typical case is a query over partitioned table where partitioning key column filter is dynamically evaluated:
SELECT ...
FROM table WHERE partition_key = (SELECT .... )
The text was updated successfully, but these errors were encountered:
For multi-step execution we could have a series of plannings: e.g:
P -> (plan and extract partial queries) -> P2, [Q1, Q2, ...Qn]
-> (plan and extract partial queries) -> P3, [Q(n+1), Q(n+2), ... Q(n+m)]
-> []
So for initial plan P we extract some partial queries Q1, ..., Qn which are fully executed. Then we replan again (with new statistics) and extract further partial queries Q(n+1), ... Q(n+m). We repeat that until we fully consume plan P (e.g: P -> P1 -> P2 -> ... -> Pi -> []).
It seems that there could be some object above SqlQueryManager that would implement such multi-stage iterative approach. For example there could be some: SqlMultiStageQueryManager that owns multiple SqlQueryManager for currently running partial query executions.
Temp tables
As part of planning we could identify which subplans quality for materialization via temp tables, e.g: via TempTablePlanNode. Such TempTablePlanNode would be natural candidates for partial queries in the iterative process above. The plan would be a DAG for moment, but that is probably not an issue.
findepi
changed the title
Multi-step execution
Phased (multi-step) execution
Aug 7, 2019
findepi
changed the title
Phased (multi-step) execution
Multi-step execution
Aug 7, 2019
Certain queries would benefit from materializing part of the query (even if only in memory) and re-planning the rest.
Typical case is a query over partitioned table where partitioning key column filter is dynamically evaluated:
The text was updated successfully, but these errors were encountered: