You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This worklog implements support for CUBE aggregation extension in
HeatWave using a transformation after the QKRN creation. CUBE is not
yet supported in MySQL, hence this functionality will be available only
in HeatWave.
What is CUBE:
==============
The CUBE group by modifier is a feature in SQL used to generate a
result set that includes all possible combinations of values for
specified columns in the GROUP BY clause. When the CUBE modifier
is used in a GROUP BY statement, it generates a result set that
includes subsets for each possible combination of values from the
specified columns. This includes all possible grouping sets, including
single column values, all combinations of two columns, three columns,
and so on.
SYNTAX:
================
GROUP BY CUBE ( <list of columns separated by comma> )
Another syntax for ROLLUP is also introduced
GROUP BY ROLLUP ( <list of columns separated by comma> )
Please note that the old syntax for ROLLUP
GROUP BY <list of columns separated by comma> WITH ROLLUP
also exists.
Execution in PRIMARY and SECONDARY engine
=========================================
The group by modifier CUBE can be executed only in Heatwave.
In case the query cannot be executed in HeatWave or the secondary
engine is switched OFF, then it cannot be executed in MySQL either
since MySQL does not support CUBE. The error message will be
"ERROR HY000: CUBE is not supported"
Constraint
================
Maximum number of Group By modifier branches to be supported is 128.
This constraint can be extracted from GetMaximumNumGrpByColsSupported.
Limitations:
==============
The existing limitations of ROLLUP also apply to CUBE which are :
1) This worklog will only support CUBE modifier for aggregations
WITHOUT DISTINCT, i.e. AGGREGATION(DISTINCT ...) will not be satisfied.
2) This worklog does NOT support CUBE modifier in the presence of
duplicate GROUP BY keys.
High level Design:
=================
Say we have a query of the form
SELECT col1, col2, SUM(data) AS summ, GROUPING(col1) as grp1,
GROUPING(col1, col2) AS grp1_2
FROM t1
GROUP BY CUBE (col1, col2);
The high-level idea is to transform a query into an equivalent form
WITH base AS (
SELECT col1, col2, SUM(data) AS summ
FROM t1 GROUP BY col1, col2
)
SELECT col1, col2, summ, 0 as grp1, 0 AS grp1_2 FROM base
UNION ALL
SELECT col1, NULL, SUM(summ), 0 as grp1, 1 AS grp1_2 FROM base
GROUP BY col1
UNION ALL
SELECT NULL, col2, SUM(summ), 1 as grp1, 2 AS grp1_2 FROM base
GROUP BY col2
UNION ALL
SELECT NULL, NULL, SUM(summ), 1 as grp1, 3 AS grp1_2 FROM base
HAVING (SELECT COUNT(*) FROM base) > 0;
Grouping set representation for ROLLUP and CUBE
==========================================================
Group by modifiers such as ROLLUP and CUBE can be represented as
grouping sets, which are determined during query preparation.
The representation of the grouping set is done using a bitfield
in the ORDER object.
case ROLLUP : Say the query has GROUP BY ROLLUP (a,b)
then the grouping sets will be (a,b) (a) ()
Here () represents full table aggregate without any grouping.
Here there are 3 grouping sets ranging from 0 to 2.
where 0 is the full table aggregate.
The bitfield associated with Group by element 'a' will be 3 (i,e. 2+1)
The bitfield associated with Group by element 'b' will be 2 as it is
part of only set number 2
case CUBE: Say the query has GROUP BY CUBE (a,b)
then the grouping sets will be (a,b) (a) (b) ()
The number of grouping sets will be (2^n) where n is the number of
elements in the GROUP BY list. The bitfield associated with
Group by element 'a' will be 6 (i,e. 4+2) The bitfield
associated with Group by element 'b' will be 1.
More implementation Details :
==================================
1) qkrn/rpd_qkrn_transform_rollup.cc is renamed to
qkrn/rpd_qkrn_transform_gb_modifier.cc to make it generic for all
group by modifiers
2) File : handler/ha_rpd_qkrn_expr.cc
Instead of checking for Qkrn_context::FOR_ROLLUP we check for all
types of group by modifiers using the flag Qkrn_context::FOR_GB_MODIFIER
3) File : sql/sql_base.cc
Fix for the crash of the following query
Query : SELECT i1 FROM t
GROUP BY CUBE (i1) HAVING (GROUPING(i1)) > 1;
Problem :The function Group_check::check_expression->find_item_in_list
attempted to find the HAVING condition item (GROUPING(i1)) > 1 in the
select list. However, find_ident field_name is nullptr for the item,
which caused the crash.
4) Bug in ha_rpd.cc
In the case of queries with inner joins, it always returned true,
indicating that the query is pushable. However, the check for constants
in the group by list was never performed for such queries.
5) File :qkrn_bugs_varlen.test
Due to a bug in IsQueryPushable, certain queries with constants in the
group by clause were mistakenly offloaded earlier.
6) File : handler/ha_rpd_qkrn_ap.cc
Previously, ExtractRequiredItemForGroupBy would return the temporary
table item, and the caller would then extract the required items from
the temporary table list as well. Now, the extraction of required items
from the temporary table is performed directly within the function.
7) File: sql/sql_lex.h
The functionality of mark_item_as_maybe_null_if_rollup_item is now
moved to Query_block::prepare.
8) File: handler/ha_rpd_qkrn_ap.cc
The call for !func->has_rollup_expr() is removed as
!HasAggregationItem(func) already performs the same check.
9) File: handler/ha_rpd_qkrn_common.cc
The function IsGroupConcatWithRollup is removed, and instead, the check
is placed within HasIncompatibleGroupConcat.
10) File : cp_func_gconcat.inc
Some of the explain statements are removed since the corresponding
select statements are already present in the file and the file is
executed with the secondary_engine=FORCED flag.
11) Bug#35669045
Query
============
SELECT alias2 . pk AS field1
FROM E AS alias2
GROUP BY CUBE (field1)
ORDER BY STRCMP( '', field1 );
In function Item_field::fix_fields, the item returned from
find_item_in_list is a Item_field, hence *reference is
NOT updated.
Solution: In function Item_field::fix_fields,
update *reference even for fields
12) Added some custom TPCH queries for comparing performance of CUBE.
13) The flag PROP_ROLLUP_EXPR is replaced with PROP_GROUP_BY_MODIFIER
14) Bug#35779978: At the end of Query_block::prepare, the Grouping set
dependency flag is set for all the items which are part of the GROUP
BY list. But the flag was not updated properly for the parent items.
Bug#35669045 - #wl15843-Segmentation fault- Rapid server crash- rpdmqxra2_bv_bitset
Bug#35683655 - #wl15843-Segmentation fault-Mysqld crash- KEY_PART_INFO::init_from_field ()
Bug#35681749 - #wl15843-Segmentation fault:-rpdmpr_bexpr_fromdays_RPDMPR_DTYPE_SB4_RPDMPR_DTYPE
Bug#35690590 - #wl15843-Rapid crash rpdmpr_bexpr_extract_RPDMPR_DTYPE_SB8_RPDMPR_DTYPE_DSB16_RP
Bug#35731905 - #wl15843-Mysqld Crash- Assertion:-rpd::CreateIndividualGroupByModifierBranch
Bug#35750113 - #wl15843:Mysqld-crash-Segmentation fault:rpd::FixColumnIdsInOpnArray
Bug#35672851 - #wl15843-Incorrect offload error messages,QCOMP: window functions failing with ROLLUP
Bug#33814459 - Exception during Qcomp, currently only support tasks with
Bug#35762767 - #wl15843-Segmentation fault:Rpdserver crash:-bv_bitset
Bug#35749317 - #wl15843-Mysqld-crash:Segmentation fault:rpd::ContainsUnsupportedCastEnumToChar
Bug#35779978 - #wl15843-Segmentation fault:Rpdserver crash:-rpdmqxra2_bv_bitset
Bug#35793146 - #wl15843-Mysqld crash-glibcxx_assert_fail-Assertion 'get() != pointer()' failed
Bug#35794723 - #wl15843-Rpdserver crash- Assertion-rpdmpr_winfunc_min_max_remove
Change-Id: I007aa396869d10f0eb787252bfd6eafc2a2a6410
SELECT product, country_id , year, SUM(profit) FROM t1
309
-
GROUP BY product, country_id, year WITH CUBE;
310
-
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WITH CUBE' at line 2
309
+
GROUP BY CUBE (product, country_id, year);
310
+
ERROR HY000: Secondary engine operation failed. No secondary engine defined for at least one of the query tables.
311
311
EXPLAIN SELECT product, country_id , year, SUM(profit) FROM t1
312
-
GROUP BY product, country_id, year WITH CUBE;
313
-
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WITH CUBE' at line 2
312
+
GROUP BY CUBE (product, country_id, year);
313
+
ERROR HY000: Secondary engine operation failed. No secondary engine defined for at least one of the query tables.
314
314
SELECT product, country_id , year, SUM(profit) FROM t1
315
-
GROUP BY product, country_id, year WITH CUBE UNION ALL
315
+
GROUP BY CUBE (product, country_id, year)
316
+
UNION ALL
316
317
SELECT product, country_id , year, SUM(profit) FROM t1
317
-
GROUP BY product, country_id, year WITH ROLLUP;
318
-
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WITH CUBE UNION ALL
319
-
SELECT product, country_id , year, SUM(profit) FROM t1
320
-
GROUP' at line 2
318
+
GROUP BY ROLLUP (product, country_id, year);
319
+
ERROR HY000: Secondary engine operation failed. No secondary engine defined for at least one of the query tables.
0 commit comments