Skip to content

[fix](nereids) fix bugs in PushDownAggThroughJoin #50846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
englefly wants to merge 1 commit into from
Closed

[fix](nereids) fix bugs in PushDownAggThroughJoin #50846

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query1.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,15 @@ PhysicalCteAnchor ( cteId=CTEId#0 )
----------hashAgg[LOCAL]
------------PhysicalProject
--------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_returns.sr_returned_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[sr_returned_date_sk]
----------------PhysicalProject
------------------PhysicalOlapScan[store_returns] apply RFs: RF0
----------------PhysicalProject
------------------filter((date_dim.d_year = 2000))
--------------------PhysicalOlapScan[date_dim]
----------------hashAgg[GLOBAL]
------------------PhysicalDistribute[DistributionSpecHash]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------PhysicalOlapScan[store_returns] apply RFs: RF0
----------------hashAgg[LOCAL]
------------------PhysicalProject
--------------------filter((date_dim.d_year = 2000))
----------------------PhysicalOlapScan[date_dim]
--PhysicalResultSink
----PhysicalTopN[MERGE_SORT]
------PhysicalDistribute[DistributionSpecGather]
Expand Down
30 changes: 19 additions & 11 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query12.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,23 @@ PhysicalResultSink
------------------PhysicalDistribute[DistributionSpecHash]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF1 i_item_sk->[ws_item_sk]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ws_sold_date_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[web_sales] apply RFs: RF0 RF1
------------------------------PhysicalProject
--------------------------------filter((date_dim.d_date <= '2001-07-15') and (date_dim.d_date >= '2001-06-15'))
----------------------------------PhysicalOlapScan[date_dim]
--------------------------PhysicalProject
----------------------------filter(i_category IN ('Books', 'Electronics', 'Men'))
------------------------------PhysicalOlapScan[item]
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF1 d_date_sk->[ws_sold_date_sk]
--------------------------hashAgg[GLOBAL]
----------------------------PhysicalDistribute[DistributionSpecHash]
------------------------------hashAgg[LOCAL]
--------------------------------PhysicalProject
----------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF0 i_item_sk->[ws_item_sk]
------------------------------------hashAgg[GLOBAL]
--------------------------------------PhysicalDistribute[DistributionSpecHash]
----------------------------------------hashAgg[LOCAL]
------------------------------------------PhysicalProject
--------------------------------------------PhysicalOlapScan[web_sales] apply RFs: RF0 RF1
------------------------------------hashAgg[LOCAL]
--------------------------------------PhysicalProject
----------------------------------------filter(i_category IN ('Books', 'Electronics', 'Men'))
------------------------------------------PhysicalOlapScan[item]
--------------------------hashAgg[LOCAL]
----------------------------PhysicalProject
------------------------------filter((date_dim.d_date <= '2001-07-15') and (date_dim.d_date >= '2001-06-15'))
--------------------------------PhysicalOlapScan[date_dim]

32 changes: 18 additions & 14 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query15.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,22 @@ PhysicalResultSink
----------PhysicalDistribute[DistributionSpecHash]
------------hashAgg[LOCAL]
--------------PhysicalProject
----------------hashJoin[INNER_JOIN shuffle] hashCondition=((catalog_sales.cs_bill_customer_sk = customer.c_customer_sk)) otherCondition=(OR[substring(ca_zip, 1, 5) IN ('80348', '81792', '83405', '85392', '85460', '85669', '86197', '86475', '88274'),ca_state IN ('CA', 'GA', 'WA'),(catalog_sales.cs_sales_price > 500.00)]) build RFs:RF2 c_customer_sk->[cs_bill_customer_sk]
------------------PhysicalProject
--------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF1 d_date_sk->[cs_sold_date_sk]
----------------------PhysicalProject
------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF1 RF2
----------------------PhysicalProject
------------------------filter((date_dim.d_qoy = 2) and (date_dim.d_year = 2001))
--------------------------PhysicalOlapScan[date_dim]
------------------PhysicalProject
--------------------hashJoin[INNER_JOIN shuffle] hashCondition=((customer.c_current_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF0 ca_address_sk->[c_current_addr_sk]
----------------------PhysicalProject
------------------------PhysicalOlapScan[customer] apply RFs: RF0
----------------------PhysicalProject
------------------------PhysicalOlapScan[customer_address]
----------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF2 d_date_sk->[cs_sold_date_sk]
------------------hashAgg[GLOBAL]
--------------------PhysicalDistribute[DistributionSpecHash]
----------------------hashAgg[LOCAL]
------------------------PhysicalProject
--------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_bill_customer_sk = customer.c_customer_sk)) otherCondition=(OR[substring(ca_zip, 1, 5) IN ('80348', '81792', '83405', '85392', '85460', '85669', '86197', '86475', '88274'),ca_state IN ('CA', 'GA', 'WA'),(catalog_sales.cs_sales_price > 500.00)]) build RFs:RF1 c_customer_sk->[cs_bill_customer_sk]
----------------------------PhysicalProject
------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF1 RF2
----------------------------PhysicalProject
------------------------------hashJoin[INNER_JOIN shuffle] hashCondition=((customer.c_current_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF0 ca_address_sk->[c_current_addr_sk]
--------------------------------PhysicalProject
----------------------------------PhysicalOlapScan[customer] apply RFs: RF0
--------------------------------PhysicalProject
----------------------------------PhysicalOlapScan[customer_address]
------------------hashAgg[LOCAL]
--------------------PhysicalProject
----------------------filter((date_dim.d_qoy = 2) and (date_dim.d_year = 2001))
------------------------PhysicalOlapScan[date_dim]

30 changes: 19 additions & 11 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query20.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,23 @@ PhysicalResultSink
------------------PhysicalDistribute[DistributionSpecHash]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF1 i_item_sk->[cs_item_sk]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[cs_sold_date_sk]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF0 RF1
------------------------------PhysicalProject
--------------------------------filter((date_dim.d_date <= '2002-07-18') and (date_dim.d_date >= '2002-06-18'))
----------------------------------PhysicalOlapScan[date_dim]
--------------------------PhysicalProject
----------------------------filter(i_category IN ('Books', 'Music', 'Sports'))
------------------------------PhysicalOlapScan[item]
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF1 d_date_sk->[cs_sold_date_sk]
--------------------------hashAgg[GLOBAL]
----------------------------PhysicalDistribute[DistributionSpecHash]
------------------------------hashAgg[LOCAL]
--------------------------------PhysicalProject
----------------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((catalog_sales.cs_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF0 i_item_sk->[cs_item_sk]
------------------------------------hashAgg[GLOBAL]
--------------------------------------PhysicalDistribute[DistributionSpecHash]
----------------------------------------hashAgg[LOCAL]
------------------------------------------PhysicalProject
--------------------------------------------PhysicalOlapScan[catalog_sales] apply RFs: RF0 RF1
------------------------------------hashAgg[LOCAL]
--------------------------------------PhysicalProject
----------------------------------------filter(i_category IN ('Books', 'Music', 'Sports'))
------------------------------------------PhysicalOlapScan[item]
--------------------------hashAgg[LOCAL]
----------------------------PhysicalProject
------------------------------filter((date_dim.d_date <= '2002-07-18') and (date_dim.d_date >= '2002-06-18'))
--------------------------------PhysicalOlapScan[date_dim]

30 changes: 19 additions & 11 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query3.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,23 @@ PhysicalResultSink
------------PhysicalDistribute[DistributionSpecHash]
--------------hashAgg[LOCAL]
----------------PhysicalProject
------------------hashJoin[INNER_JOIN broadcast] hashCondition=((dt.d_date_sk = store_sales.ss_sold_date_sk)) otherCondition=() build RFs:RF1 d_date_sk->[ss_sold_date_sk]
--------------------PhysicalProject
----------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF0 i_item_sk->[ss_item_sk]
------------------------PhysicalProject
--------------------------PhysicalOlapScan[store_sales] apply RFs: RF0 RF1
------------------------PhysicalProject
--------------------------filter((item.i_manufact_id = 816))
----------------------------PhysicalOlapScan[item]
--------------------PhysicalProject
----------------------filter((dt.d_moy = 11))
------------------------PhysicalOlapScan[date_dim]
------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_item_sk = item.i_item_sk)) otherCondition=() build RFs:RF1 i_item_sk->[ss_item_sk]
--------------------hashAgg[GLOBAL]
----------------------PhysicalDistribute[DistributionSpecHash]
------------------------hashAgg[LOCAL]
--------------------------PhysicalProject
----------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((dt.d_date_sk = store_sales.ss_sold_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ss_sold_date_sk]
------------------------------hashAgg[GLOBAL]
--------------------------------PhysicalDistribute[DistributionSpecHash]
----------------------------------hashAgg[LOCAL]
------------------------------------PhysicalProject
--------------------------------------PhysicalOlapScan[store_sales] apply RFs: RF0 RF1
------------------------------hashAgg[LOCAL]
--------------------------------PhysicalProject
----------------------------------filter((dt.d_moy = 11))
------------------------------------PhysicalOlapScan[date_dim]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------filter((item.i_manufact_id = 816))
--------------------------PhysicalOlapScan[item]

26 changes: 17 additions & 9 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query30.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,23 @@ PhysicalCteAnchor ( cteId=CTEId#0 )
----------hashAgg[LOCAL]
------------PhysicalProject
--------------hashJoin[INNER_JOIN shuffle] hashCondition=((web_returns.wr_returning_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF1 ca_address_sk->[wr_returning_addr_sk]
----------------PhysicalProject
------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_returns.wr_returned_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[wr_returned_date_sk]
--------------------PhysicalProject
----------------------PhysicalOlapScan[web_returns] apply RFs: RF0 RF1
--------------------PhysicalProject
----------------------filter((date_dim.d_year = 2000))
------------------------PhysicalOlapScan[date_dim]
----------------PhysicalProject
------------------PhysicalOlapScan[customer_address]
----------------hashAgg[GLOBAL]
------------------PhysicalDistribute[DistributionSpecHash]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_returns.wr_returned_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[wr_returned_date_sk]
--------------------------hashAgg[GLOBAL]
----------------------------PhysicalDistribute[DistributionSpecHash]
------------------------------hashAgg[LOCAL]
--------------------------------PhysicalProject
----------------------------------PhysicalOlapScan[web_returns] apply RFs: RF0 RF1
--------------------------hashAgg[LOCAL]
----------------------------PhysicalProject
------------------------------filter((date_dim.d_year = 2000))
--------------------------------PhysicalOlapScan[date_dim]
----------------hashAgg[LOCAL]
------------------PhysicalProject
--------------------PhysicalOlapScan[customer_address]
--PhysicalResultSink
----PhysicalTopN[MERGE_SORT]
------PhysicalDistribute[DistributionSpecGather]
Expand Down
56 changes: 36 additions & 20 deletions regression-test/data/shape_check/tpcds_sf1000/shape/query31.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,48 @@ PhysicalCteAnchor ( cteId=CTEId#0 )
------PhysicalDistribute[DistributionSpecHash]
--------hashAgg[LOCAL]
----------PhysicalProject
------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF1 ca_address_sk->[ss_addr_sk]
--------------PhysicalProject
----------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ss_sold_date_sk]
------------------PhysicalProject
--------------------PhysicalOlapScan[store_sales] apply RFs: RF0 RF1
------------------PhysicalProject
--------------------filter((ss.d_year = 1999) and d_qoy IN (1, 2, 3))
----------------------PhysicalOlapScan[date_dim]
--------------PhysicalProject
----------------PhysicalOlapScan[customer_address]
------------hashJoin[INNER_JOIN bucketShuffle] hashCondition=((store_sales.ss_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF1 ss_addr_sk->[ca_address_sk]
--------------hashAgg[LOCAL]
----------------PhysicalProject
------------------PhysicalOlapScan[customer_address] apply RFs: RF1
--------------hashAgg[GLOBAL]
----------------PhysicalDistribute[DistributionSpecHash]
------------------hashAgg[LOCAL]
--------------------PhysicalProject
----------------------hashJoin[INNER_JOIN broadcast] hashCondition=((store_sales.ss_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF0 d_date_sk->[ss_sold_date_sk]
------------------------hashAgg[GLOBAL]
--------------------------PhysicalDistribute[DistributionSpecHash]
----------------------------hashAgg[LOCAL]
------------------------------PhysicalProject
--------------------------------PhysicalOlapScan[store_sales] apply RFs: RF0
------------------------hashAgg[LOCAL]
--------------------------PhysicalProject
----------------------------filter((ss.d_year = 1999) and d_qoy IN (1, 2, 3))
------------------------------PhysicalOlapScan[date_dim]
--PhysicalCteAnchor ( cteId=CTEId#1 )
----PhysicalCteProducer ( cteId=CTEId#1 )
------hashAgg[GLOBAL]
--------PhysicalDistribute[DistributionSpecHash]
----------hashAgg[LOCAL]
------------PhysicalProject
--------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_bill_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF3 ca_address_sk->[ws_bill_addr_sk]
----------------PhysicalProject
------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF2 d_date_sk->[ws_sold_date_sk]
--------------------PhysicalProject
----------------------PhysicalOlapScan[web_sales] apply RFs: RF2 RF3
--------------------PhysicalProject
----------------------filter((ws.d_year = 1999) and d_qoy IN (1, 2, 3))
------------------------PhysicalOlapScan[date_dim]
----------------PhysicalProject
------------------PhysicalOlapScan[customer_address]
--------------hashJoin[INNER_JOIN bucketShuffle] hashCondition=((web_sales.ws_bill_addr_sk = customer_address.ca_address_sk)) otherCondition=() build RFs:RF3 ws_bill_addr_sk->[ca_address_sk]
----------------hashAgg[LOCAL]
------------------PhysicalProject
--------------------PhysicalOlapScan[customer_address] apply RFs: RF3
----------------hashAgg[GLOBAL]
------------------PhysicalDistribute[DistributionSpecHash]
--------------------hashAgg[LOCAL]
----------------------PhysicalProject
------------------------hashJoin[INNER_JOIN broadcast] hashCondition=((web_sales.ws_sold_date_sk = date_dim.d_date_sk)) otherCondition=() build RFs:RF2 d_date_sk->[ws_sold_date_sk]
--------------------------hashAgg[GLOBAL]
----------------------------PhysicalDistribute[DistributionSpecHash]
------------------------------hashAgg[LOCAL]
--------------------------------PhysicalProject
----------------------------------PhysicalOlapScan[web_sales] apply RFs: RF2
--------------------------hashAgg[LOCAL]
----------------------------PhysicalProject
------------------------------filter((ws.d_year = 1999) and d_qoy IN (1, 2, 3))
--------------------------------PhysicalOlapScan[date_dim]
----PhysicalResultSink
------PhysicalQuickSort[MERGE_SORT]
--------PhysicalDistribute[DistributionSpecGather]
Expand Down
Loading