Logical error 'Unexpected return type from equals' in group_by_use_nulls + GROUPING SETS

[vdimir]

https://fiddle.clickhouse.com/dba51b74-4843-46e5-abbc-8cb5d0310373

And it works on latest, though. But broken on head

DROP TABLE IF EXISTS test_grouping_sets_predicate;

CREATE TABLE test_grouping_sets_predicate
(
    day_ Date,
    type_1 String
)
ENGINE=MergeTree
ORDER BY day_;

INSERT INTO test_grouping_sets_predicate SELECT toDate('2023-01-05') AS day_, 'hello, world' FROM numbers (10);

SET group_by_use_nulls = true;

SELECT *
FROM
( SELECT
    day_,
    type_1
FROM test_grouping_sets_predicate
GROUP BY
    GROUPING SETS ( (day_, type_1), (day_) )
)
WHERE day_ = '2023-01-05';

Code: 49. DB::Exception: Received from localhost:9000. DB::Exception: Unexpected return type from equals. Expected Nullable. Got UInt8: while executing 'FUNCTION equals(day_ : 0, '2023-01-05' :: 1) -> equals(day_, '2023-01-05') Nullable(UInt8) : 3'. (LOGICAL_ERROR)

[antonio2368]

just a note, it was probably broken before 24.1 but @Algunenano added a check for it which is getting triggered now
#59379

[azat]

One more - https://s3.amazonaws.com/clickhouse-test-reports/60695/048a042dc4963631a23358d3e454dcd8a9eaafa2/stress_test__debug_.html

[divanik]

One more - https://s3.amazonaws.com/clickhouse-test-reports/59390/7d38789e3053b6912300e9fd48456cd3b3426690/stress_test__tsan_.html
Also what interesting here is that there is no stack trace in clickhouse-server.stress.log

[Avogar]

The problem here is in filter pushdown:

create table test (x UInt32, y UInt32) engine=MergeTree order by x;
insert into test values (1, 2), (2, 3);

With pushdown:

select * from (select * from test group by grouping sets ((x, y), (y))) where y = 2 settings group_by_use_nulls=1, query_plan_filter_push_down=1;

<Fatal> : Logical error: 'Unexpected return type from equals. Expected Nullable. Got UInt8'

Without pushdown:

select * from (select * from test group by grouping sets ((x, y), (y))) where y = 2 settings group_by_use_nulls=1, query_plan_filter_push_down=0;

┌────x─┬─y─┐
│ ᴺᵁᴸᴸ │ 2 │
└──────┴───┘
┌─x─┬─y─┐
│ 1 │ 2 │
└───┴───┘

Let's see result of explain plan for both:

:) explain plan header=1, actions=1 select * from (select * from test group by grouping sets ((x, y), (y))) where y = 2 settings group_by_use_nulls=1, query_plan_filter_push_down=1;

EXPLAIN header = 1, actions = 1
SELECT *
FROM
(
    SELECT *
    FROM test
    GROUP BY
        GROUPING SETS (
            (x, y),
            (y))
)
WHERE y = 2
SETTINGS group_by_use_nulls = 1, query_plan_filter_push_down = 1

Query id: d3e446c1-8313-492b-9f35-04ef0e2aba28

Execute function equals/equals(y, 2)/Nullable(UInt8)
┌─explain───────────────────────────────────────────────────────────────────────────────┐
│ Expression ((Projection + (Before ORDER BY + )))                                      │
│ Header: x Nullable(UInt32)                                                            │
│         y Nullable(UInt32)                                                            │
│ Actions: INPUT :: 0 -> x Nullable(UInt32) : 0                                         │
│          INPUT :: 1 -> y Nullable(UInt32) : 1                                         │
│ Positions: 0 1                                                                        │
│   Aggregating                                                                         │
│   Header: __grouping_set UInt64                                                       │
│           x Nullable(UInt32)                                                          │
│           y Nullable(UInt32)                                                          │
│   Keys: x, y                                                                          │
│   Skip merging: 0                                                                     │
│     Filter                                                                            │
│     Header: x UInt32                                                                  │
│             y UInt32                                                                  │
│     Filter column: equals(y, 2) (removed)                                             │
│     Actions: INPUT : 1 -> y Nullable(UInt32) : 0                                      │
│              COLUMN Const(UInt8) -> 2 UInt8 : 1                                       │
│              INPUT :: 0 -> x UInt32 : 2                                               │
│              FUNCTION equals(y : 0, 2 :: 1) -> equals(y, 2) Nullable(UInt8) : 3       │
│     Positions: 3 2 0                                                                  │
│       Expression                                                                      │
│       Header: x UInt32                                                                │
│               y UInt32                                                                │
│       Actions: INPUT :: 0 -> x UInt32 : 0                                             │
│                INPUT :: 1 -> y UInt32 : 1                                             │
│       Positions: 0 1                                                                  │
│         ReadFromMergeTree (default.test)                                              │
│         Header: y UInt32                                                              │
│                 x UInt32                                                              │
│         ReadType: Default                                                             │
│         Parts: 1                                                                      │
│         Granules: 1                                                                   │
│         Prewhere info                                                                 │
│         Need filter: 1                                                                │
│           Prewhere filter                                                             │
│           Prewhere filter column: equals(y, 2) (removed)                              │
│           Actions: INPUT : 0 -> y UInt32 : 0                                          │
│                    COLUMN Const(UInt8) -> 2 UInt8 : 1                                 │
│                    FUNCTION equals(y : 0, 2 :: 1) -> equals(y, 2) Nullable(UInt8) : 2 │
│           Positions: 0 2                                                              │
└───────────────────────────────────────────────────────────────────────────────────────┘

:) explain plan header=1, actions=1 select * from (select * from test group by grouping sets ((x, y), (y))) where y = 2 settings group_by_use_nulls=1, query_plan_filter_push_down=0;

EXPLAIN header = 1, actions = 1
SELECT *
FROM
(
    SELECT *
    FROM test
    GROUP BY
        GROUPING SETS (
            (x, y),
            (y))
)
WHERE y = 2
SETTINGS group_by_use_nulls = 1, query_plan_filter_push_down = 0

Query id: 069ea73f-98a0-4b2b-bdb0-a38de0116752

Execute function equals/equals(y, 2)/Nullable(UInt8)
┌─explain─────────────────────────────────────────────────────────────────────────┐
│ Expression ((Projection + Before ORDER BY))                                     │
│ Header: x Nullable(UInt32)                                                      │
│         y Nullable(UInt32)                                                      │
│ Actions: INPUT :: 0 -> x Nullable(UInt32) : 0                                   │
│          INPUT :: 1 -> y Nullable(UInt32) : 1                                   │
│ Positions: 0 1                                                                  │
│   Filter ((WHERE + (Projection + Before ORDER BY)))                             │
│   Header: x Nullable(UInt32)                                                    │
│           y Nullable(UInt32)                                                    │
│   Filter column: equals(y, 2) (removed)                                         │
│   Actions: INPUT :: 0 -> x Nullable(UInt32) : 0                                 │
│            INPUT : 1 -> y Nullable(UInt32) : 1                                  │
│            COLUMN Const(UInt8) -> 2 UInt8 : 2                                   │
│            FUNCTION equals(y : 1, 2 :: 2) -> equals(y, 2) Nullable(UInt8) : 3   │
│   Positions: 0 1 3                                                              │
│     Filter (HAVING)                                                             │
│     Header: x Nullable(UInt32)                                                  │
│             y Nullable(UInt32)                                                  │
│     Filter column: equals(y, 2) (removed)                                       │
│     Actions: INPUT :: 0 -> x Nullable(UInt32) : 0                               │
│              INPUT : 1 -> y Nullable(UInt32) : 1                                │
│              COLUMN Const(UInt8) -> 2 UInt8 : 2                                 │
│              FUNCTION equals(y : 1, 2 :: 2) -> equals(y, 2) Nullable(UInt8) : 3 │
│     Positions: 0 1 3                                                            │
│       Aggregating                                                               │
│       Header: __grouping_set UInt64                                             │
│               x Nullable(UInt32)                                                │
│               y Nullable(UInt32)                                                │
│       Keys: x, y                                                                │
│       Skip merging: 0                                                           │
│         Expression (Before GROUP BY)                                            │
│         Header: x UInt32                                                        │
│                 y UInt32                                                        │
│         Actions: INPUT :: 0 -> x UInt32 : 0                                     │
│                  INPUT :: 1 -> y UInt32 : 1                                     │
│         Positions: 0 1                                                          │
│           ReadFromMergeTree (default.test)                                      │
│           Header: x UInt32                                                      │
│                   y UInt32                                                      │
│           ReadType: Default                                                     │
│           Parts: 1                                                              │
│           Granules: 1                                                           │
└─────────────────────────────────────────────────────────────────────────────────┘

With pushdown we see:

│           Prewhere filter                                                             │
│           Prewhere filter column: equals(y, 2) (removed)                              │
│           Actions: INPUT : 0 -> y UInt32 : 0                                          │
│                    COLUMN Const(UInt8) -> 2 UInt8 : 1                                 │
│                    FUNCTION equals(y : 0, 2 :: 1) -> equals(y, 2) Nullable(UInt8) : 2 │

When we use group_by_use_nulls=1, we make all keys from grouping sets Nullable, but seems like during pushdown we don't check that and as a result, in filter that was pushed down we have non-nullable input column for equals function but expect Nullable result from it.

[Avogar]

Let's just disable filter pushdown with grouping sets and group_by_use_nulls=1

[vdimir]

In current head reproducer from the first comment returns Illegal column for DataTypeNullable. (ILLEGAL_COLUMN) about __table1.day_ Nullable(Date) UInt16(size = 0) instead of logical error, but still doesn't look correct

UPD: for allow_experimental_analyzer=0 it's still logical error